San Francisco local officials are questioning the safety of autonomous vehicles (AV), also known as self-driving cars, due to frequent reports of traffic violations, delaying public transportation and emergency vehicles, causing traffic congestion as well as driving onto public sidewalks.
California officials granted the first AV deployment permits this year, allowing companies to release self-driving cars onto city streets and to provide passenger service as robotaxis. Since granting permits to AV companies such as Waymo and Cruise, self-driving vehicles have been creating a multitude of issues with public safety, including a recent incident of a Cruise AV stopping a bus on its route for over seven minutes.
As state governments have the legal power to grant permits to AV companies to conduct testing and ride hail services, city officials are left powerless to control self-driving car incidents that affect residents and public safety. City officials also struggle to obtain information regarding AV-induced road blockages and even a comprehensive list of all companies have deployed self-driving vehicles onto their roads.
Advances in GNSS technology constantly expand the range of projects that benefit from them.
ComNav Technology
A telecom company adopted its CORS station to build China’s national CORS service for public companies. It is increasingly used for field robotics, including the development of self-driving cars.
Leica Geosystems
Bernhard Richter, vice president of Geomatics, Leica Geosystems AG, pointed to one of the biggest infrastructure projects in Europe, which aims to connect London to Birmingham, Manchester and Leeds with a high-speed railway system, avoiding the need to fly between those cities. This will have great environmental benefits because high-speed trains are much more efficient than planes.
However, high-speed rail requires tremendous precision. “First comes the prep work, moving dirt,” said Richter. “Then you must install the railroad ties with tenths of a millimeter precision relative to each other to avoid side accelerations. For a surveyor, it really has everything in one project. You need to constantly work with civil engineers. You then try to build as much as possible with machine-control-guided systems to make the leveling as automated as possible.” The project will include building bridges over whole valleys and monitoring them, particularly during the construction phase, to ensure that they are not moving.
“Even the factory they are building is huge, so just to build the factory you need a lot of surveying,” Richter said. The project is generating 25,000 jobs at 300 construction sites, all of which must be managed on very tight schedules. In this context, the quality of the survey gear is critical. “On a construction site, the surveyor should be an invisible person,” Richter said. “When they come with the big machines and want to get stuff done, they don’t want a surveyor on the site. So, he has to work off hours, then remain on alert and trust that what comes out of an instrument is correct.” Leica Geosystems is one of the main suppliers for this project. “They chose us because of our focus on reliability, trust and quality.”
Trimble
Software is increasingly driving sales, pointed out Boris Skopljak, vice president, Surveying & Mapping Strategy and Product Marketing at Trimble Inc. As an example, he cited Trimble’s SX12 scanning total station, which uses Trimble Access software to leverage scanning, imaging and traditional total station capabilities in the field. “We have provided more inspection tools to enable people to decide whether something is meeting the tolerance.” The Trimble Connect cloud-based collaboration platform, coupled with the continuous field and office connectivity, has driven productivity increases and moved customers toward choosing the company’s solutions, he said.
As an example of Trimble solutions, Skopljak cited City Rail Link, New Zealand’s first underground rail network and the largest transportation infrastructure project ever undertaken there. “The Trimble R10 was integral to acquiring static observations above the work site, while the Trimble S9, DiNi and Trimble Business Center network adjustment were game changers for the survey control network,” he said. To expedite mine tunneling the surveyors used the SX12’s combined total station and scanning functionality with Trimble Access field software infield inspection tools. “Fewer customers are choosing solutions on a spec. It’s not about how many satellites you can track, for how many days, or how many points you can scan. They are choosing solutions based on the ecosystem and productivity.”
Tesla has applied to patent a new multi-band GNSS antenna with increased precision for self-driving and navigation in its autos, reports electrek. The antenna would be positioned inside the rear-mirror enclosure, and the system would also provide a heating element.
While increasing precision, it is also designed to reduce power demand, noise and cost.
The automaker describes the new device in the abstract of the new patent application:
A multi-band antenna system is provided. The antenna system can be placed under and embedded within a glass exterior surface of a vehicle. Such an antenna system can include a capacitively coupled metallic element on or adjacent to the glass exterior surface, which can serve as both a parasitic element to enhance gain and as a heating element to melt snow and/or ice accumulation over the glass area that covers the antenna. In certain applications, the antenna’s structure itself can be used as a heater to improve performance in adverse weather conditions while the heating elements are positioned away from the thermally sensitive electronics. The antenna system with integrated heating can include a spiral antenna.
A diagram of the antenna provided with the patent application. (U.S. Patent Office)
How inertial systems and GNSS availability will help
By Kana Nagai, Matthew Spenko, Ron Henderson and Boris Pervan
Self-driving cars in urban environments can be problematic. The required multi-sensor automated systems will include GNSS, but buildings block and reflect GNSS signals, reducing system availability and accuracy. Researchers from the Illinois Institute of Technology report on how inertial navigation systems coupled with wheel-speed sensors and vehicle dynamic constraints can help.
Innovation Insights with Richard Langley
ARE WE THERE YET? This was a familiar refrain from the backseats of parents’ cars when traveling to a holiday destination or to grandparents when I was growing up. We didn’t have videos on a display attached to the seats in front of us or (who could imagine?) our own personal communication device on which we could call up games, movies or social media channels.
But I’m not talking about that complaint from our childhoods. I’m asking if we have arrived at the era of the self-driving car. The answer is yes and no. It all depends on what you mean by “self-driving.” We reviewed some of the technologies needed for self-driving or autonomous vehicles in this column in June 2019. And we indicated in the introduction to that column that vehicle autonomy has several levels. SAE International, formerly known as the Society of Automotive Engineers, has defined six levels of autonomy that can be briefly described as Level 0 – no automation; Level 1 – hands on/shared control; Level 2 – hands off; Level 3 – eyes off; Level 4 – mind off; and Level 5 – steering wheel optional.
Already, Level 1 automation is widely available in modern cars with adaptive cruise control, parking assistance, lane-keeping assistance and automatic emergency braking among the features being offered.
Level 2 automation, where the automated system takes full control of the vehicle’s acceleration, braking and steering, is available in some production models, although the “hands-off” designation is not to be taken literally — most motor vehicle laws require drivers to keep their hands on the steering wheel.
Between Level 2 and Level 3, we have conditional automation — the car can drive itself, but the driver must stay alert and be prepared to take over immediately.
Level 3 is high automation, where a computer fully drives the car at certain times on certain routes such as a highway; while the driver can perform other tasks such as reading a book, they must be prepared to take over operation of the vehicle within a few seconds if alerted by the automated system. While test campaigns are still ongoing, some jurisdictions permit Level 3 operation by ordinary drivers on some roads, and customers will soon be able to buy vehicles with this level of automation. Widespread use of
Level 4 and Level 5 automation is further off (some would say quite a way off) and remains in development. But famously, last year, Toyota operated Level 4 self-driving shuttle vehicles around the Tokyo 2020 Olympic Village.
A lot more work needs to be done before we will have arrived at the era of the fully self-driving car that will be able to travel on any road, anywhere in the world, all year around, in all weather conditions. In particular, self-driving cars in urban environments (as opposed to highway driving) can be problematic.
The required multi-sensor automated systems will include GNSS, but buildings block and reflect GNSS signals, reducing system availability and accuracy. In “Innovation” this month, researchers from the Illinois Institute of Technology report on how inertial navigation systems coupled with wheel-speed sensors and vehicle dynamic constraints can help.
GNSS provides navigation services globally, but satellite visibility in urban areas is limited by high-rise buildings. This creates a mixture of GNSS available and denied environments (see FIGURE 1) — users do not generally know where the system can maintain sufficient levels of accuracy and integrity for a particular application. To begin to address the issue for self-driving cars, we evaluated GNSS-only availability in downtown Chicago.
FIGURE 1. The figure depicts three types of potential GNSS signal reception: direct LOS signals and blocked LOS signals (left) and reflected LOS signals (right). (Image: Authors)
GNSS signal prediction in urban environments has been conducted in previous work. For example, the concept of “shadow matching” was developed to identify GNSS signal blockages in urban canyons. Overlaying sky plots on a hemispherical sky view can be used to distinguish between line-of-sight (LOS) and non-line-of-sight (NLOS) signals (see FIGURE 2a). Reflected rays can be predicted using Householder transformations to reveal potential multipath conditions. Satellites producing blocked or reflected (NLOS) signals should be excluded to maintain integrity.
FIGURE 2. (a) A hemispherical sky view in an urban environment. (b) Illustration of a protection level and an alert limit. To ensure integrity, the protection level must not exceed an alert limit. (c) The allowable probability of exceedance is assumed to be 10−7 in this work. (Image: Authors)
When the number of visible satellites is greater than three, GNSS can resolve vehicle position. However, even in cases where enough satellites are visible, the satellite geometries are generally weak because the dilution of precision (DOP) is adversely affected by the buildings partially blocking the sky. Horizontal positioning error must be bounded by a protection level computed by the vehicle. Then, for navigation to be deemed available, the protection level must not exceed a required alert limit (see FIGURE 2b). The maximum allowed probability of exceedance (see FIGURE 2c) and the alert limit can together be used to determine the maximum allowable position error standard deviation.
Even if the protection level is far below the alert limit in an open-sky environment, it will frequently exceed the alert limit once the vehicle enters a city. GNSS alone is generally not able to maintain availability, so integration with other sensors is needed. Tightly coupling inertial navigation systems (INS) with GNSS using the extended Kalman filter (EKF) provides better estimation in urban environments. The EKF algorithm also enables integration of wheel-speed sensors and vehicle dynamic constraints. These integrated navigation systems will improve availability, but it is still unclear how long such a system can be expected to maintain fault-free integrity in a congested city.
Focusing on the problem of self-driving cars in urban environments, we evaluate protection levels of navigation with practical integrated sensors: GNSS, INS, a wheel-speed sensor (WSS) and vehicle dynamic constraints (VDC). The goal is to develop the means by which we can determine locations where external ranging sources (such as lidar) are needed to maintain continuous navigation with fault-free integrity.
GNSS-ONLY AVAILABILITY
For GNSS availability evaluation, we assume an integrity requirement that the probability of exceeding a 0.5-meter alert limit must be lower than 10−7. The 0.5-meter alert limit therefore corresponds to approximately five times the position standard deviation, so the maximum allowable position error standard deviation is then approximately 0.1 meters. Accuracy at this level clearly requires differential GNSS carrier-phase measurements. We assume a nominal GNSS double difference (DD) carrier ranging error standard deviation of approximately 0.02 meters, and that carrier cycle ambiguities can be readily resolved in an open-sky environment prior to initiation of vehicle motion.
Given the assumptions made of the maximum allowable position error standard deviation and the GNSS ranging error standard deviation, the maximum allowable horizontal dilution of precision (HDOP) is about 5.
FIGURE 3 shows GPS and GNSS availability — the fraction of time the HDOP requirement is met over 24 hours — along a section of State Street in downtown Chicago. The availability results using GPS only and excluding only blocked LOS signals ranged from 0% to 9% along the block and 9% to 30% at the intersections (see FIGURE 3a). Using four full GNSS constellations (GPS, Galileo, GLONASS and BeiDou), availability ranged from 48% to 82% along the block and 72% to 100% at the intersections (see FIGURE 3b).
FIGURE 3. The percentage of GPS or GNSS availability in 3D-mapped downtown Chicago. We exclude satellites producing blocked LOS signals or both blocked and reflected LOS (NLOS) signals from the measurements. Each column expresses a lane of southbound or northbound travel. The availability is the percentage of total time when HDOP meets the self-driving car integrity requirements in 24 hours. (Image: Authors)
When we also excluded satellites producing reflected LOS signals that reach the vehicle, the availability dropped significantly at every point (see FIGURE 3c). We assert that FIGURE 3c expresses the reality of GNSS availability because building-reflected multipath signals degrade positioning accuracy and would affect integrity negatively. It’s obvious from these results that GNSS alone is insufficient to meet the autonomous driving requirements in an urban environment, and multi-sensor integrated navigation systems are needed to augment poor GNSS signal availability.
MULTI-SENSOR INTEGRATION
We begin by considering tightly coupled INS/GNSS integration using an EKF, and then integrate a realistic sensor suite including WSS and vehicle dynamic constraints that enforce resistance to lateral sliding and vertical movement. If it is known from another source that the vehicle is not moving (for example, it is in the parking gear), a static mode constraint (SMC) can also be applied.
INS/GNSS Integration. Tightly coupled INS/GNSS integration with an EKF uses the INS measurement to predict vehicle motion. The continuous process model uses a state vector having the position in the navigation frame, the velocity, the attitude, bias errors and cycle ambiguities, with the input vector having accelerometer-specific force measurement in the body frame and gyro-rotation-rate measurements. A white-noise vector drives the inertial measurement unit (IMU) states.
The GPS/GNSS measurement model includes the measurement vector having carrier and code phases, and the observation matrix containing LOS vectors and the vector of white receiver thermal noise.
INS/GNSS/WSS/VDC Integration. For the vehicle in motion, we developed a model consisting of a WSS measurement in the along-track direction, a non-holonomic constraint resisting lateral sliding, and a holonomic constraint on vertical movement (see FIGURE 4).
The INS/GNSS/WSS/VDC integration using the EKF consists of the process model and the measurement models.
FIGURE 4. The measurement model consisting of the WSS measurement in the along-track direction (vx), non-holonomic constraint resisting lateral sliding (vy), and holonomic constraint on vertical movement (vz). N is the navigation frame, Ac is the rear-axle center point and Bc is the center point of the body-fixed frame. (Image: Authors)
INS/GNSS/SMC Integration. The static mode constraint provides zero-velocity measurements to the EKF measurement update to mitigate position error propagation. We use SMC only when it is known that the vehicle is not moving; for example, when the vehicle is in the parking gear.
Error Propagation Analysis. We tested the time from perfect initialization to when position error exceeds 0.1 meters in GNSS-denied environments. FIGURE 5 shows the error growth in the along-track (x), the cross-track (y) and the vertical (z). The error specifications for a STIM300 tactical-grade IMU are used in this analysis. The standard deviation of the WSS measurement noise is assumed to be 0.05 meters per second, and the standard deviation of the movement constraint violations is 0.001 meters per second. The vehicle is moving at 5 meters per second except when we test the SMC.
The INS can coast 15.6 seconds before the position error standard deviation exceeds 0.1 meters in both the along-track and the cross-track directions (see FIGURE 5a). The INS/WSS/VDC can coast 16.5 seconds in the along-track direction, and significantly more than 40 seconds (the simulation duration) in the cross-track direction (see FIGURE 5b). In static mode, INS/SMC estimate errors do not grow with time in any direction, as expected (see FIGURE 5c). In GNSS-denied environments, the non-holonomic constraint suppresses the cross-track position error, but the WSS measurement hardly affects the along-track position error. The SMC works perfectly, but the usage is limited to when the vehicle is known to be stationary.
FIGURE 5. The vehicle position error growth vs. time in the along-track (x), cross-track (y) and vertical (z) directions. Each graph represents the navigation system introduced in the multi-sensor integration section. The vehicle is moving at 5 meters per second (a and b) or 0 meters per second (c). (Image: Authors)
SIMULATION SCENARIO
We imagine a future driverless-car mission scenario in which multi-sensor navigation systems are practicable. To minimize congestion in a city, autonomous vehicles will be held outside the urban core when not in use. In the clear open-sky environment, a vehicle in a parking lot completes GNSS initialization using the INS/GNSS/SMC system. Once requested for action, the vehicle departs for the city from the parking lot, and the motion of the vehicle improves alignment by the INS/GNSS system. Safe navigation can be ensured using the system to provide continuity under overpasses and bridges in the open-sky environment. Upon entering the urban core, navigation becomes more dependent on the INS/WSS/VDC system.
A reasonable numerical target for differential GNSS initialized position error is 0.02 meters, and for the INS alignment yaw angle error 0.1 degrees.
Local GNSS multipath errors from nearby vehicles will vary with the satellite elevation angle. Prior experimental results show that lower elevation-angle satellite signals (below 33 degrees) are much more likely to be impacted by multipath than higher ones (see TABLE 1).
Table 1. The nominal GNSS multipath error values in the simulation.
INITIALIZATION AND ALIGNMENT
Initialization takes place in a parking lot with a clear sky view. A vehicle is in the parking gear, enabling SMC to be applied. FIGURE 6a shows a typical example: with INS/GPS/SMC, system initialization takes about 31 minutes, and with INS/GPS, about 36 minutes. Therefore, SMC does speed up GPS initialization, although the improvement is modest.
The yaw angle is not aligned during the initialization, but roll and pitch are immediately aligned (see FIGURE 6b). Earth’s gravity affects roll and pitch angle alignment but not yaw angle.
Yaw angle alignment cannot be performed when the vehicle is stationary or moving with constant velocity. Accelerated motion, either straight or turning, is required.
FIGURE 6. (a) Comparisons of initialization time between INS/GPS and INS/GPS/SMC in an open-sky environment. The INS/GPS/SMC system initializes rapidly. (b) Transitions of roll, pitch, yaw alignment during the initialization. Yaw angle alignment cannot be performed when the vehicle is stationary. (Image: Authors)
FIGURE 7 shows the behavior of the yaw angle error standard deviation using the INS/GPS system when centripetal (see FIGURE 7a) or tangential (see FIGURE 7b) acceleration is applied. The yaw angle can be aligned in a couple of seconds for either type of acceleration. To represent typical initial motions of self-driving cars, we model a parking-lot departure via a “Z”-shaped path. In this scenario, the yaw alignment error reaches 0.1 degrees within a couple of seconds (see FIGURE 7c).
FIGURE 7. The behavior of yaw angle error when centripetal (a) or tangential (b) acceleration is applied; (c) shows the behavior while following a z-shaped path. The yaw angle can be aligned in a couple of seconds in each case. (Image: Authors)
EVALUATION IN URBAN ENVIRONMENTS
After initialization and alignment in the open-sky environment, we simulated the vehicle traveling into the urban core. The urban environment in our study is 3D-mapped State Street in Chicago, which runs north-south and transits from low-rise neighborhoods to central downtown. We selected one congested section surrounded by tall buildings and computed the position error standard deviation along the path. The evaluation points are at 10-meter intervals over a total distance of 170 meters. The yellow lines in FIGURE 8 denote the visible satellites, identified by their pseudorandom noise (PRN) code numbers, at each point. We assume for convenience that the INS/GPS system is initialized and aligned at the first evaluation point. In reality, we would expect a degraded initial condition because we are starting the simulation in an urban canyon.
FIGURE 8. Evaluation points and PRN numbers of visible satellites at each point. (Image: Authors)
In the first simulation, the car equipped with the INS/GPS system moved either 1 or 5 meters per second. The y-axis in FIGURE 9 represents the position error standard deviation, and the x-axis represents the distance in meters. The dotted line expresses the number of visible satellites. The error when the vehicle velocity is 1 meter per second exceeded the maximum allowable position error standard deviation of 0.1 meter, at the distance of 60 meters. However, when the velocity was 5 meters per second, the maximum allowable position error standard deviation was never reached. It is also clear from the figures that error propagation is significantly affected by the number of visible satellites.
FIGURE 9. A comparison of position error growth between velocities of 1 meter per second and 5 meters per second. (Image: Authors)
In the second simulation, we compared two different navigation systems, INS/GPS and INS/GPS/WSS/VDC. The vehicle moved at 1 meter per second in the same urban environment. The INS/GPS/WSS/VDC system does provide relief, but the error propagation is still clearly affected by the number of visible satellites (see FIGURE 10).
FIGURE 10. A comparison of position error growth between the INS/GPS and INS/GPS/WSS/VDC systems for a velocity of 1 meter per second. (Image: Authors)
In GNSS-challenged environments, INS error propagation is a function of time. When a vehicle moves faster, it clears the blockage area more quickly, reducing the impact of INS drift — a function of time, not distance. In contrast, GNSS error is completely determined by location. Because INS error propagation depends on how long the vehicle stays in an area of GNSS outage, protection levels for trips through the same area will be different if the vehicle is smoothly cruising or gets stuck in a traffic jam.
CONCLUSION
To gain a better understanding of how long and under what local conditions multi-sensor integrated navigation systems can maintain fault-free integrity, we evaluated navigation positioning errors in 3D-mapped downtown Chicago. The system we developed consists of sensors with which self-driving cars would reasonably be equipped: GNSS, INS, WSS and dynamic constraints. We showed that INS/GPS position errors along the path depend very strongly on the vehicle’s speed. When the system is augmented with WSS/VDC, position errors are suppressed, but the error propagation is still strongly influenced by the number of visible satellites.
ACKNOWLEDGMENTS
The research described in this article is supported by the National Science Foundation. Figure 1 was created by Alexis Arias of the Landscape Architecture + Urbanism Program at the Illinois Institute of Technology (IIT). The authors greatly appreciate the advice and help of Nilay Mistry from that program.
This article is based on the paper “Evaluating INS/GNSS Availability for Self-Driving Cars in Urban Environments” presented at ION ITM 2021, the virtual 2021 International Technical Meeting of The Institute of Navigation, Jan. 25–28, 2021.
KANA NAGAI is a Ph.D. candidate and research assistant in mechanical and aerospace engineering at IIT.
MATTHEW SPENKO is a professor of mechanical and aerospace engineering at IIT. He earned his M.S. and Ph.D. degrees in mechanical engineering from the Massachusetts Institute of Technology.
RON HENDERSON is a professor and director of the Landscape Architecture + Urbanism Program at IIT. He earned his Master of Landscape Architecture and Master of Architecture from the University of Pennsylvania.
BORIS PERVAN is a professor of mechanical and aerospace engineering at IIT. He earned his M.S. from the California Institute of Technology and Ph.D. from Stanford University.
Years ago, a trucker driving down the western slope of the Rocky Mountains lost his brakes. As his truck accelerated, he hoped to make it to the next runaway truck ramp before losing control. However, when he reached it, he saw a car parked at its base with a group of teenagers drinking beers. In a split-second decision, he veered to the left instead and went off the cliff. In the coming years, faced with the same moral dilemma, what would a self-driving truck do?
Matteo Luccio
Many similar scenarios have been discussed in the technical literature on self-driving vehicles. Most of them are variations on the “trolley problem” presented to generations of college philosophy students since it was first formulated by philosopher Philippa Foot in 1967 and adapted by Judith Jarvis Thomson in 1985. In the trolley problem, a person can choose to divert a trolley from the main track, saving five people who are working on it but killing a person on the other track who otherwise would not have been involved.
When faced with an inevitable crash, should a self-driving car slam into a wall to save the lives of three children crossing the street or, in effect, target them to save its two occupants? Most people, when polled, choose the former. When shopping for a new car, however, those same people are more likely to buy one that will make their own safety its highest priority.
Human drivers react to emergencies instinctively — motivated by neither forethought nor malice — and in real time. By contrast, the choices made by autonomous vehicles are predetermined by programmers; their control systems can potentially estimate the outcome of various options within milliseconds and take actions that factor in an extensive body of research, debate and legislation. Therefore, our judgment is harsh if those vehicles make what we deem to be the “wrong” choice.
However, there is no universal agreement as to what constitutes the “right” choice, other than the fact that people generally prefer self-driving cars to minimize the number of lost lives and to privilege people over animals and younger people over older ones. General principles such as “to minimize harm” are of little help in complex and dynamic real-life situations.
Self-driving cars, in addition to their many other benefits, will dramatically reduce traffic accidents and fatalities, because they will never be distracted, drowsy, drunk or drugged. Yet accidents will still happen, and their outcomes will be largely determined far in advance.
The mass introduction of self-driving cars onto public roads will require overcoming technical, legal and ethical challenges. As a society, we will have to agree on a uniform set of ethical codes that will guide these vehicles’ decision-making processes in emergencies. This will force us to explicitly quantify the value of human life and property, and encode it in software. These are hard and uncomfortable choices.
Autonomous systems, fusing data from multiple sensors, will guide these vehicles. It is up to us to decide whom they will target and whom they will spare.
In June 2019, Regulus Cyber’s experts successfully spoofed the GPS-based navigation system of a Tesla Model 3 vehicle. This experiment provided an important warning for all companies using GNSS location and timing: these technologies, on which they depend, are highly vulnerable to spoofing attacks. In the two years since the experiment, companies and governments have continued to research the potential harm that can be caused by spoofing attacks and are learning more about how to defend themselves from them.
The Tesla experiment was groundbreaking because it was the first time that a level 2.5 autonomous vehicle was exposed to a sophisticated GPS spoofing attack and its behavior recorded.
We chose Tesla’s Model 3 because it had the most sophisticated advanced driver assistance system (ADAS) at the time, called Navigate on Autopilot (abbreviated NOA or Autopilot), which uses GPS to make several driving decisions. However, this experiment exposed several cybersecurity issues potentially affecting all vehicles relying on GPS as part of their sensor fusion for autonomous decision making.
NOA makes lane changes and takes interchange exits once a destination is determined, without requiring any confirmation by the driver. Its several other features include autonomous deceleration and acceleration according to the speed limit, autonomous lane changing, and adaptive cruise control.
These features use a variety of sensors, including cameras, radar, speedometers and more. The researchers wanted to test the extent to which the Model 3 relied on its GNSS receiver to make these driving decisions and how it behaved when receiving contradicting information from its GNSS receiver and its other sensors.
The researchers used hardware and software purchased online to mimic the tools potential hackers would use. The experiment involved two software-defined radio (SDR) devices purchased online, one to spoof GPS and one to jam all other constellations, connected to an external antenna to simulate an external attack. The software used to simulate the GPS signal was downloaded from an online source, available for free.
The test included three scenarios the researchers assumed would involve usage of GNSS, each one using a different spoofing pattern:
Scenario 1. Exiting the highway at the wrong location
Scenario 2. Enforcing an incorrect speed limit
Scenario 3. Turning into incoming traffic
A Tesla Model 3 was remotely hacked in a test of a GPS spoofing attack. (Photo: Regulus Cyber)
Scenario 1: Exiting the Highway at the Wrong Location
The car was driving normally at a constant speed of 95 KPH with NOA enabled. The destination determined for this ride was a town nearby and the car designated a certain interchange as the destination for an autonomous exit maneuver. The experiment began 2.5 km before the vehicle reached that interchange; however, the researchers’ fake GPS signal resulted in coordinates of a location on the same highway but only 150 m before the exit.
As soon as its GNSS receiver was spoofed, the car assumed that it had reached the correct exit and began to maneuver to the right, activating the blinker, slowing down, turning the wheel, and crossing a dotted white line to its right side, exiting to an emergency pit-stop, confusing it with the exit 2.5 km ahead.
To be clear, this would not have happened at any location along the highway, because sensor fusion with the radar and the camera enables the car to avoid physical obstacles and ensures that it does not cross a solid white line that makes a turn illegal.
The spoofing attack succeeded, in that it enabled the attacker to remotely manipulate the car’s sensor fusion and make it exit the highway at the wrong location.
Scenario 2: Enforcing an Incorrect Speed Limit
The car was driving to a random city far away on a highway, at a constant speed of 90 KPH, which was 10 KPH below the highway’s speed limit, with NOA enabled. The researchers generated a fake GPS signal, with the coordinates of a nearby town road that has a speed limit of 33 KPH. Shortly thereafter, the vehicle assumed the speed limit had just changed to 33 KPH and instantly began decelerating. Each time the driver attempted to accelerate using the gas pedal, as soon as he lifted his foot off the pedal the car engaged in heavy braking to quickly decelerate back to 33 KPH.
To be clear, this would not have happened if NOA had been turned off. The cruise mode can be disabled by either using the touch screen or by pressing the brakes, which would allow the driver to regain full manual control over the vehicle’s speed.
Again, the spoofing attack succeeded, in that it allowed the attacker to remotely manipulate the car’s speed and made it enforce a speed limit much lower than the actual one on the highway.
Scenario 3: Turning into Incoming Traffic
The car was being driven manually on a two-lane road with one lane in each direction, the type of road on which NOA cannot be used. The researchers generated a fake GPS signal, with coordinates of a nearby three-lane highway, with all lanes in the same direction. Furthermore, the spoofed location was 150 m from a designated exit that the vehicle’s navigation system was programmed to take, requiring a left turn.
Shortly after the car’s GNSS receiver was spoofed, the vehicle assumed it was on a highway and engaged NOA. Next, it triggered the exit maneuver, which began with activating the left blinker, followed by turning the wheel to the left. The driver had to quickly grab the wheel and manually drive the car back to its lane to avoid a collision with oncoming traffic.
To be clear, this kind of scenario would not be possible without the driver enabling the NOA. Once a Tesla driver enables NOA, it automatically turns on once the vehicle is on the highway with a set destination. This is why the researchers assumed that NOA would be turned on by default, and as long as NOA is activated, the vehicle is susceptible to the attacks mentioned in the experiment.
Once again, the spoofing attack was successful in that it enabled the attacker to remotely steer the vehicle into the opposing lane, placing it on a direct collision course with oncoming traffic. Out of the three scenarios described, this one proved that GNSS spoofing can endanger lives.
The hardware used for the GPS spoofing test. (Photo: Regulus Cyber)
GPS Cybersecurity for Automotive Applications
The NOA system in the Tesla Model 3, being an ADAS, allows drivers to rely on the car and its sensors for basic driving functions. Therefore, it enables drivers to briefly take their hands off the wheel and reduces the number of actions they are required to take. Nevertheless, drivers are still required to be fully attentive to the road so that they can take control of the vehicle at any time.
However, since this spoofing attack had such a sudden and instant impact on the car’s driving behavior, a driver who is not fully attentive and aware would not be prepared to quickly take control and prevent an accident. By the time the driver notices that something is wrong and reacts, it might be too late to prevent an accident. Already drivers have been found sleeping at the wheel, driving under the influence of alcohol, and doing other inappropriate tasks with NOA engaged.
Furthermore, this situation assumes a level 2.5 autonomous vehicle as was tested. But what happens in level 3 vehicles, in which driver engagement is limited, or level 4 and 5, in which driver response is non-existent? This research provides us with a glimpse into the crucial importance of sensor cybersecurity and particularly of GNSS cybersecurity.
The Tesla hack experiment and its results were eye-opening for the autonomous vehicles sector – the danger is real and rising as more and more vehicles are depending on GNSS technology as part of their sensors for assisted or automated driving. Up to 97% of new vehicles since 2019 incorporate GNSS receivers and most if not all are still vulnerable to the same spoofing attacks presented in this research.
In January 2021, the UN’s World Forum for Harmonization of Vehicle Regulations (WP.29) issued Regulation No. 155, which sets guidelines for cybersecurity in the automotive industry with the goal of addressing every possible cyber threat that it might encounter. Annex 5 of the regulation defines cyber attacks and states that in order to get approvals in the future vehicle manufacturers will need to provide solid evidence that their vehicles are sufficiently protected against them.
Among the cyber threats mentioned in the Annex is spoofing of data received by the vehicle — both sybil spoofing attacks and spoofing of messages. The Annex also lists the appropriate protection that vehicle manufacturers should implement and states that vehicle manufacturers will be required to provide evidence of the effectiveness of the mitigation measures they choose. These upcoming regulatory requirements can make the difference between life and death in situations caused by GNSS spoofing and ensure that only reliable and resilient positioning is used within vehicles, both today and in the future.
Please note: Tesla released a statement saying that it is “taking steps to introduce safeguards in the future which we believe will make our products more secure against these kinds of attacks.” Regulus Cyber researchers did not perform any further experiments with Tesla Model 3 since this research was published two years ago.
See the Tesla GPS spoofing experiment from the driver’s point of view:
The TomTom autonomous test vehicle. (Photo: TomTom)
To grab a larger piece of the burgeoning connected and autonomous vehicle markets, digital mapmaker TomTom launched its self-driving test vehicle and is integrating navigation and diagnostic capabilities in the Microsoft Connected Vehicle Platform.
While the company has tested the technology for years, TomTom officially announced the availability of its own autonomous test vehicle, which Arnold Meijer, TomTom strategic manager, business development, said is a critical advantage to quickly develop maps and services for that industry.
“We can continuously test our mapping technology on the roads, get insights and high-quality data on how it performs in a multitude of circumstances and, right away, feed this into our AI-driven mapmaking process,” he said.
Currently, the vehicle is testing TomTom’s high-definition map; a crowd-sourced map update called Roadagrams; and a map-streaming service, AutoStream, which will deliver map data to vehicles on demand.
The company announced in September at the International Motor Show (IAA 2019) in Frankfurt, Germany, that its digital maps have doubled to more than 1 million Level 1 and Level 2 autonomous vehicles from several automakers.
Showcasing TomTom HD Maps
At IAA 2019, TomTom also demonstrated proof-of-concept high-definition map features that can work through Microsoft Azure cloud services, Meijer said. “Some of the navigation intelligence, which also includes traffic information and HD map services, can also be used in vehicles for navigation apps, as well as context while driving autonomously,” he said.
The demonstration allows TomTom to send telemetry data to Microsoft Azure. “[This includes] application usage data such as what menu items the end-user clicked on and what screens of the TomTom NavApp they opened for app analytics purposes,” he said.
Other features include GPS location during a driving simulation, vehicle speed and heading for driver behavior, and usage of apps that are not navigation-related, such as for music. Other testing included road speed limit, the number of times users drive a planned route, destination details planned by a user, and travel time.
TomTom navigation integrated into the Microsoft Connected Vehicle Platform allows automakers to quickly access precise navigation and driving behavior, “while of course adhering to TomTom’s privacy principles,” Meijer said. “This data could, for instance, be used to predict the range of an electric vehicle based on driving behavior and planned route more accurately. Or to work out, based on navigation behavior, what connectivity package for online navigation would be best suited for a driver.”
The TomTom Moma test vehicle prototype hits the road in San Francisco. (Photo: Kevin Dennehy)
Voyage Forges Ahead…
While recent press reports have said autonomous vehicle development is slowing because of technology limitations and consumer doubt, Palo Alto, Calif.-based Voyage has raised millions of dollars and continues to test self-driving cars in retirement communities in California and Florida.
The company recently raised $31 million in Series B funding from Franklin Templeton, Khosla Ventures, Jaguar Land-Rover’s InMotion Ventures and Chevron Technology Ventures. Voyage has raised a total of $52 million since its 2017 founding, said Oliver Cameron, company co-founder and CEO.
With the new funding, Cameron said the company hopes to triple its workforce, increase its second-generation fleet of self-driving cars, invest more in the technology and roll out a third-generation vehicle.
“We are taking a unique, focused approach to delivering truly driverless cars in communities where there are limited mobility options, customers who need an autonomous ride-hailing service, and lower speeds,” he said. “Many residents within our communities don’t have access to transportation options that work for them, so they’ve welcomed our fleet of self-driving vehicles. We’ve started with self-driving cars that can travel point-to-point within our communities at speeds of up to 25 mph.”
Cameron said the community-based approach allows the company to quickly design and deploy autonomous technology. “For the past two years, because we are taking this focused, community-based approach to designing and deploying advanced self-driving car technology, progress has been rapid. Our vehicles intelligently and autonomously navigate the complex neighborhoods of our communities and safely transport our passengers door-to-door,” he said.
The company said its engineers are transitioning software to a safety-critical middleware, shipping a new prediction engine with better capabilities and creating triple redundancy in the vehicle’s perception system.
G3 Vehicle? Not Yet. Voyage, which began testing self-driving with a retrofitted Ford Fusion, doesn’t have a timetable planned for a third-generation system, Cameron said. Instead, the company plans to continue to use the self-driving Chrysler Pacifica hybrid minivan in its testing projects.
“Before we launch our [third generation] self-driving car, we are going to build and scale more G2 self-driving cars to better serve the communities in which we operate. We haven’t announced a timeframe for the launch of our electric, truly driverless and highly scalable G3 self-driving car,” he said.
The company is growing its ranks as it has hired its first chief operations officer, Nina Qi. She believes the biggest hurdle autonomous vehicles have is the basic challenge to deliver safe and cost-effective services to make the industry economically sustainable.
In July, Volkswagen AG and Ford Motor Co. provided updates on their development alliance announced in January.
The automakers plan to collaborate on autonomous vehicles, among other programs. Together, they are investing $2.6 billion in Pittsburgh startup Argo AI, which is developing a self-driving technology platform. Ford first invested in Argo two years ago.
The investment includes the resources of VW’s Autonomous Intelligent Driving Group (AID), valued at $1.6 billion. AID will become Argo AI’s European operation.
Volkswagen and Ford hope to achieve a self-driving platform that can be scaled comparatively quickly. Argo AI’s objectives are to
build for scale.
architect the software to be production quality.
have automotive-grade sensors and computers.
fully integrate their product with OEMs and automakers.
A benefit to having the Argo AI system on more vehicles means the AI will obtain data through daily operation, enabling it to grow smarter and better.
Argo AI has successfully tested its driverless vehicles in five U.S. cities: Pittsburgh, Palo Alto, Detroit, Miami and Washington, D.C.
A man took a video of his 70-year-old mother’s reaction to a Tesla Model S on Autopilot. Sitting in the driver’s seat, the woman screams, pleading with her son to put her back in control of the vehicle.
