Lidar sensors that used to cost tens of thousands of dollars now cost only hundreds of dollars. With prices significantly decreasing, 3D sensors are more accessible than ever before. Now, what was once a niche technology exclusively for autonomous vehicles is being deployed globally to make places safer and smarter. Additionally, the industry is continuing to grow: market analysis firm Yolé estimates that the lidar industry will be worth nearly $4 billion by 2025, a 19% CAGR between 2020 and 2025.
While decreasing sensor prices are a critical factor in the proliferation of lidar, an arguably more significant development is the development of robust perception software that can track, identify and monitor with far greater accuracy and efficiency than ever before.
Effective 3D sensors, from lidar to radar and 3D cameras, require both hardware and software components. The hardware is critical to capturing data with high resolution and accuracy, while the software processes and comprehends the data, making them actionable. Essentially, software is the “brain” of sensors. Lidar, without equally strong perception software, is like an iPhone without iOS — inoperable and just a piece of machinery.
Today, at the confluence of these factors, we are beginning to see a proliferation of 3D perception applications beyond autonomous driving. Cities, security and retail are a few key sectors where I predict we will continue to see advancements over the next few years.
Making Cities Smarter
The steep drop in the cost of lidar sensors has made 3D sensors more accessible than ever. (Image: Seoul Robotics)
Today’s cities have a variety of challenges to address, from decreasing traffic collisions to reducing congestion, and we are witnessing municipalities leveraging lidar to collect critical insights into city safety and efficiency.
However, why are they turning to 3D solutions, specifically? Because they can capture the data necessary to make actionable changes. 3D sensors were developed to quickly track and analyze city surroundings for autonomous vehicles, so they are an effective way to ensure that vehicles are not veering into opposing lanes or traversing crosswalks already occupied by pedestrians.
Cities also adopt 3D applications because they can often address multiple challenges with one system. For example, a city may install a lidar system on an intersection to detect traffic violations, but the system can also capture data about pedestrian safety and traffic flow. These multi-benefit solutions are ultimately more cost-effective for cities because they eliminate the need to install multiple different solutions to solve these problems.
Creating Safer Spaces
Companies are turning to 3D data to create safer and more secure environments. (Image: Seoul Robotics)
From airports to museums, from stadiums to music venues, the market for 3D-based security solutions is vast. While each of these environments is unique in how it operates, they all rely on technology to ensure that areas are secure, visitors do not enter prohibited areas, and crowds are seamlessly moving through the space.
3D perception helps address these challenges by creating “zones” that can alert security systems if someone enters. Additionally, because 3D sensors can detect and track various objects, including humans, they are increasingly becoming a popular solution for crowd control. They can help venues monitor and address foot traffic, such as with security lines, and they can be valuable in the event of an emergency to ensure that an area is clear.
Beyond the tangible benefits 3D sensors bring to different venues, companies are turning to 3D data to create safer and more secure environments because they are more accurate and anonymous. Unlike traditional camera-based systems such as CCTV, which are often prone to false positives, 3D data are incredibly accurate and precise, so they are less likely to set off alarms unnecessarily. Additionally, 3D data do not include biometric information, so they address privacy concerns while still ensuring that areas are secure.
Building 3D Retail Environments
By implementing 3D-based solutions into a physical retail environment, companies can better understand how shoppers are moving through and spending their time in stores. They can glean insights into key metrics, such as:
How long are people in line?
What areas of the store are receiving the most traffic?
With what products are people engaging most frequently?
As one example, Mercedes-Benz has integrated 3D sensors into its showrooms in Korea, gaining fascinating insights into customer behavior. For example, they’ve discovered that nearly 60% of customers spend their time looking at the trunk space of SUVs, and that red is the most popular color.
As these solutions continue to become more sophisticated and accessible, we should expect to see them in more areas of our everyday lives. The future of 3D perception is exciting, and it will ensure safer, smarter and more efficient spaces — improving the quality of life.
HanBin Lee is CEO of Seoul Robotics, a 3D perception company specializing in lidar.
After years of testing and hype, not a lot of companies can say there are real applications for autonomous technology. However, at this year’s virtual CES 2021 trade show, both Caterpillar and John Deere, two companies known for their tractors and heavy equipment, showcased autonomous machines that are being used worldwide in farming and mining projects.
Photo: Caterpillar
Deerfield, Ill.-based Caterpillar, a first-time exhibitor at CES this year, said it has been involved in autonomy and use of GPS for more than two decades. “We were an early adopter of GPS when there were few satellites in the sky,” said Denise Johnson, company group president, resource industries. “We have 350 autonomous trucks operating 24-7 on three continents.”
The company’s autonomous vehicles, in addition to other technology, are being used around the clock in the Kearl Oil Sands project in Alberta, Canada.
“We are using autonomy primarily in mining operations in harsh environments. These [vehicles] are operating 24-7, with no loss time incidents,” said Bill Dears, Caterpillar worldwide sales and marketing manager. “We also track people underground with cameras and radar.”
In addition to production enhancement, safety is a factor in mining operations because of operator fatigue — something that is precluded by autonomous mining equipment, Dears said.
Agriculture uses variety of sensors, including GNSS
To Moline, Ill.-based John Deere, exhibiting at the trade show for the third time, agriculture is a high-tech industry that uses GPS, self-driving tractors, artificial intelligence and a multitude of sensors. The company rolled out its first self-driving tractors nearly 20 years ago, said Jahmy Hindman, John Deere CTO.
Photo: John Deere
The company won the CES Innovation Award for one of its tractor and combine product lines. “Both our planter and tractor have GPS and antennas to know where to drive and where exactly fertilizer [is to be placed],” Hindman said. “These tractors are self-propelled, with accuracy augmented with [real-time kinematic] sub-inch accuracy for the planters in a field.”
Among other requirements, Hindman said that tractors have to drive in a straight line, plant the required amount seeds and position them at the right depth. “When a tractor drives in a very straight line, the burden is off of the farmer. The yields increase—this is the way we see the progression of automation,” he said. “We are excited about 5G and its lower latency and high bandwidth. It opens up a lot of opportunity.”
Organizers roll out Indy Autonomous Challenge race car
At the virtual CES, representatives from the Indy Autonomous Challenge unveiled the Dallara IL-15 race car that will be used in a head-to-head race around the famous Indianapolis Motor Speedway on Oct. 23.
The Indy Autonomous Challenge, organized by Energy Systems Network and Indianapolis Motor Speedway, pits 500 university students, developing autonomous vehicle technology, against each other for a $1.5 million prize.
Logo: Indy Autonomous Challenge
Organizers say the speeds are estimated to be as much as 200 mph around the 2.5-mile track, for 20 laps, which enables researchers to evaluate how autonomous vehicle technology works in extreme conditions. They say that the goal of the race is to advance the implementation of autonomous vehicles and advanced driver-assistance systems (ADAS), much like the 2005 Defense Advanced Research Projects Agency (DARPA) Grand Challenge.
The race track has been the scene of much innovation throughout the years, said Doug Boles, Indianapolis Motor Speedway president. “Firestone tests tire technology there and that data transfers to our cars. One of the first conversations we had with Roger Penske [after Penske Entertainment bought the speedway] was about the autonomous challenge,” he said.
IAC sponsors include ADLINK, Ansys, Aptiv, AutonomouStuff, Bridgestone, CU-ICAR, Dallara, Indiana Economic Development Corp., Microsoft, New Eagle, PWR, RTI, Schaeffler and Valvoline.
Mobileye plans to test autonomous fleets in four cities
Intel subsidiary Mobileye plans to launch autonomous vehicle fleet testing in Detroit, Paris, Shanghai and Toyko. The announcement, made at CES by CEO Amnon Shashua, said that the company also plans to test in New York City, pending regulatory approval.
The company also plans to use in-house-built lidar sensors, while continuing to champion its camera-based testing. “We are using crowd-sourced data through the Cloud to build high-definition maps at scale,” Shashua said. “Thousands of product vehicles are sending us data.”
Shashua addressed a moderator’s question that cameras alone cannot be the technology of choice for autonomous vehicles. “The camera first is crucial from a technology and business point of view. We have to find out what is acceptable failure for Level 4 autonomy. Camera-only is ideal, but pushing the envelope for driver-assistance systems,” he said. “Consumer AV will take place in the 2025 timeframe. [Eventually], we can build lidar and radar to the same performance levels as camera systems. Lidar and radar can be added later for redundancy, but only for Level 4.”
Shashua said getting to Level 4 could take a decade, but that would be unsustainable unless there are government-funded projects to keep companies afloat. “By 2025, a subsystem will be good enough for consumers. Regulation is critical and sometimes it’s difficult to leap to a consumer level,” he said.
Not everyone believes what Mobileye is testing constitutes “driverless” status. To Alain Kornhauser Princeton University professor and transportation program director, who was head of the university’s team during the 2005 DARPA Challenge, not many companies are capable of full driverless capability.
“Unfortunately, I still see all of this as simply ‘eye candy’ to sell something that actually has no intention of delivering what it is implying. I still claim that the business case is zero, doesn’t exist, for personally-owned autonomous vehicles,” Kornhauser said in his Smart Driving Cars weekly newsletter. “Mobileye is nowhere close to being able to operate safely on most roads, let alone all roads. Thus, the consumer market has zero opportunity to scale.”
Kornhauser said that driverless testing is being conducted only in one place, Phoenix, by Waymo. “Neither Tesla nor Mobileye are driverless anywhere. They both require on-board human driver supervision,” he said. “That’s why they are only self-driving [tests].”
In other CES news:
GM CEO Mary Barra unveiled a single-seat electric vertical takeoff and landing (eVTOL) concept aircraft. The aircraft will be developed for future use as an air taxi. Barra briefly mentioned that the company’s Super Cruise self-driving technology will be integrated into 22 car models in a few years. The company also rolled out an electric vehicle for deliveries that can travel 250 miles on a charge and a motorized pallet for deliveries that can be tracked.
Photo: Mercedes-Benz
The Mercedes-Benz’ MBUX Hyperscreen, rolled out at CES, evaluates map data, surroundings and provides information about landmarks along a route, said Sajjad Khan, company CTO and member of the board of management. The new map feature, called Mercedes Travel Knowledge, allows a passenger or driver to ask a question as they drive by a landmark (“hey, Mercedes, what can you tell me about this building?”). The MBUX Hyperscreen is available in the new S-Class cars.
HERE Technologies introduced a mapping-as-a-service platform at CES. The platform is targeted to businesses wanting to create custom map datasets for advanced analytics and services, the company said. Some use cases include industrial yard mapping, leveraging probe data from private vehicle fleets in order to create or update a map.• A virtual CES is hard to get used to. After more than 20 years of covering the massive trade show in person, covering press conferences and conducting interviews online was sometimes a challenge. Sometimes the press conferences did not have question-and-answer sessions, or canned answers given to executives by public relations people. This doesn’t happen much during an in-person interview. In addition, trying to chat with “booth” personnel online was cumbersome and often those requests for information were ignored.
A roundup of recent products in the GNSS and inertial positioning industry from the August 2018 issue of GPS World magazine.
OEM
IP Solution
With multi-constellation GNSS for internet of things (IOT) devices
The Dragonfly NB2 is a highly integrated and modular IP (internet protocol) solution optimized for Cat-NB2 (3GPP Release 14 eNB-IoT) that can seamlessly be incorporated into chips and modules by the multitude of companies looking to address the large and fast-growing cellular IoT space. GNSS hardware package. For customers developing NB-IoT products that also require GNSS capabilities, Ceva-Dragonfly NB2 includes a new power-optimized GNSS hardware package, with GNSS RF receiver and multi-constellation digital front-end. The GNSS package speeds up both acquisition and tracking tasks by up to 8 times compared to Ceva-Dragonfly NB1, enabling a host of popular NB-IoT use cases, including people, livestock and asset tracking and geofencing.
Provides timing accuracy and stability when GNSS signal is lost
Photo: Oscilloquartz
Oscilloquartz has launched its enhanced primary reference time clock (ePRTC) system to enable a high level of timing accuracy and stability, even when the GNSS signal is lost. The system provides a timing source for mission-critical transport systems, such as utility networks, government infrastructure and radio access networks, and provides the strict synchronization needed for LTE-A and 5G applications. Featuring the OSA 3230B ePRC atomic cesium clock connected to an Oscilloquartz clock combiner and grandmaster, the new solution offers the extremely stable frequency of a cesium clock with the UTC-traceable signal provided by GNSS. When combined with the OSA 5430, the OSA ePRTC system provides full hardware redundancy and multiple fan-out options including PTP over 10 Gbit/s.
The SE878Kx-A series of GPS and GNSS integrated antenna receiver modules offer high performance, maximum reliability and low power consumption for consumer and business applications. The SE878K3-A and SE878K7-A are compatible with GPS, GLONASS, Beidou and Galileo and also enable device vendors to develop quickly and cost-effectively location-based IoT solutions for use in virtually any country worldwide. The SE878Kx-A series supports dual internal-external antennas to ensure connectivity when one is broken or compromised, along with a SAW filter to maximize jamming immunity. The modules are designed for mission-critical applications and other use cases where reliability is key, such as alarms, stolen cars or high-end asset tracking. The series also provides seamless integration with Telit’s cellular modules, including eCall/ERA-GLONASS compliant solutions.
The Spresence main and extension boards are designed for internet of things (IoT) applications. The main board uses a multi-CPU structure equipped with Sony’s GNSS receiver (GPS+GLONASS) and high-resolution audio codec. A variety of systems for applications such as drones and other IoT devices can be built by combining the boards and developing the relevant applications. The boards’ software and hardware is available via open platform, allowing for a wide range of developmental possibilities. The main board can be used to control a drone using GPS positioning and a high-performance processor, voice-controlled smart speakers and low-power consumption sensing cameras. It also can be combined with sensors for use in systems that detect errors in production lines on the factory floor.
Designed for geopositioning, construction and mapping
Photo: Topcon
The T-18 handheld controller has a 3.7-inch sunlight-readable display, a 1-GHz processor and 1 GB of internal storage. For field data collection using Topcon’s MAGNET software, the T-18 offers a durable ergonomic solution with fast processing, excellent connectivity and a long (10-hour) battery life. It has a 3.5G cellular modem for connectivity with Topcon MAGNET solutions for sending and receiving data to the cloud company account. The modem also can be used for real-time kinematic (RTK) correction services. Other features include Bluetooth and an IP65 rating for dust and water protection in demanding job-site conditions.
The SXblue ToolBox is an Android application for SXblue GNSS receivers, enabling users to view and analyze the position data and metadata related to its location. The user can send commands that enable or disable some features, including systems in use, mask angle or differential angle, and constellation in use, including GPS, GLONASS, Galileo, BeiDou and SBAS. The SXblue ToolBox is also an NTRIP client capable of connecting to a NTRIP server for real-time kinematic (RTK) corrections, allowing the receiver to issue very accurate location information. The application can record, save and transfer raw data from the GNSS receiver, allowing post-processing on computers for surveying and geomatics professionals.The toolbox has been developed with special consideration for modern mobile devices and attention to user and dealer feedback. It includes a series of configurable audible and visual alarms for determining the thresholds of the information provided by the SXblue GNSS receiver.
The Leica RTC360 laser scanner is equipped with edge computing technology to enable fast and accurate creation of 3D models in the field. It combines high-performance laser scanning, edge computing and mobile app technologies to preregister captured scans quickly and accurately. With the push of a button, two million points per second of high dynamic range imagery can be captured to create a full-dome scan in under two minutes. It features a visual inertial system that automatically tracks movements between setup positions. The scans captured can be combined and preregistered on a mobile device, where they can be viewed and augmented with information tags.
Location technology allows users to see rooms, gates and offices
Screenshot: Esri
ArcGIS Indoors is designed to enable interactive indoor mapping of corporate facilities, retail and commercial locations, airports, hospitals, event venues, universities and more. The solution applies the latest location technology to allow users to see and share where assets, rooms, departure gates and offices are located. It uses data streams, real-time processing and location intelligence tools to help businesses and other organizations understand how to better coordinate space and other resources with their facilities and campuses. Insights from sensor networks deliver real-time information to managers and executives through interactive dashboards, while visitors and employees can find useful information about the buildings they occupy. The solution also allows users to quickly access and explore critical business information, such as the location and status of fire extinguishers and their last inspection dates.
Meets demands for continuous, accurate vehicle location
The ASM330LHH module. (Photo: STMicroelectronics)
The automotive-grade ASM330LHH six-axis inertial sensor is designed for super-high-resolution motion tracking in advanced vehicle navigation and telematics applications. It lets advanced dead-reckoning algorithms calculate precise position from sensor data if satellite signals are blocked, such as in urban canyons, tunnels, covered roadways, parking garages or dense forests. Its advanced, low-noise, temperature-stable design enables dependable telematics services such as e-tolling, tele-diagnostics and e-Call assistance. Precision inertial data in six axes also meets the needs of advanced automated-driving systems. Automotive component manufacturer Magneti Marelli has selected the ASM330LHH for advanced telematics systems, to be fitted as original equipment by global automotive groups in upcoming vehicle ranges.
Esri and Waze smart cities partnership grows. (Image: Esri)
The free crowdsourced traffic and navigation app Waze is now fully supported by ArcGIS Online, where its live feed of mapped traffic alerts and other information, such as accidents, congestion and street damage, can be used in applications in minutes. Waze Live Alerts, available in ArcGIS Marketplace, is free to members of the Waze Connected Citizens Program. The program, a two-way sharing of publicly available traffic and road condition information, offers governments a stream of data, constantly updated in real time. This enables personnel to make data-driven infrastructure decisions and improves the efficiency of incident response.
Traffic engineers can use the data to analyze problems on the road and create targeted solutions.
Open-source platform for autonomous delivery and other iot
The AGL platform provides Mercedes-Benz Vans with the ability to create autonomous delivery robots. (Image: Daimler)
Automotive Grade Linux (AGL) is a collaborative cross-industry effort to develop an open platform for the connected car. Mercedes-Benz vans are using AGL as a foundation for a new onboard operating system for its commercial vehicles. The Mercedes-Benz “adVANce” initiative focuses on connectivity and internet of things (IoT) applications, innovative hardware solutions, new on-demand mobility and rental concepts, and fleet management solutions. The AGL platform provides Mercedes-Benz Vans with the flexibility to rapidly create tailored solutions for customers, including adding and connecting any kind of IoT component to the vehicle, such as sensors, automation controls and actuators. The new AGL-based operating system will debut on various Mercedes-Benz Vans prototype projects later this year.
eCyber is an integrated hardware-software product that protects vehicles against ransomware and other cyber-attacks. It can be installed in a vehicle by authorized parties, such as vehicle importers and fleet managers, in the aftermarket stage after the vehicle has left the factory, as well as by the OEM itself during manufacture. eCyber, a combined hardware and software solution in a compact box, is installed between the vehicle’s external communications device and the vehicle’s CAN (Controller Area Network) bus. It provides a secure gateway for outside communications to the CAN bus, allowing only communications with predefined parameters and values to go through. It blocks any unrecognized communications to and from the CAN bus, so no malicious digital communications can disrupt vehicle function.
Engineered for UAV-imaging missions, the iXM 100MP is a high-productivity metric camera with a range of high-resolution lenses. It is ready for integration with various UAV platforms, including Phase One’s DJI Matrice 600 Pro. The camera incorporates a medium-format sensor with backside-illumination technology, enabling high light sensitivity and extended dynamic range. Phase One also offers four new RSM lenses — with focal lengths ranging from 35mm to 150mm — to fit the new sensor’s 3.76 μm pixel size and 33 x 44 mm frame size. The lenses are available with either fixed-focus or motorized-focus functionality. The fixed-focus 35mm and 80mm lenses are especially suitable for surveying applications.
For quick approval of flights over controlled airspace
Screenshot: Skyward
Commercial drone operators in California and Hawaii — as well as a few areas in Nevada, Utah and Arizona — can get quickly authorized to fly in controlled airspace using the LAANC (Low Altitude Airspace Notification Capability) platform. Skyward is an FAA-approved airspace vendor. With Skyward, pilots with a Part 107 license can get permission to fly in regulated airspace in seconds compared to manual authorizations that can take months. This makes it significantly easier for businesses of all sizes, particularly in the construction and warehousing industries, to manage a fleet of drones to access valuable, cost-saving data. Skyward’s LAANC expansion includes airspace in the busy metro areas of Los Angeles, the Bay Area, San Diego, Las Vegas and more than 50 smaller air markets.
VP says company remains “bullish”on in-car Wi-Fi, sees need for both embedded, bring-your-own-device solutions
Mercedes-Benz has been in the news in the past month for offering connected car service as standard for five years on all new vehicles, one of the longest multi-year offers yet. But it isn’t just big news for Mercedes; it’s also a testament to Verizon’s continued investment in connected car technology.
Verizon is one of the longest tenured telematics providers in the U.S. market and has worked with Mercedes-Benz since 2009, said Mike Peterson, vice president and general manager of OEM business for Verizon Telematics, in an exclusive interview with GPS World.
“Mbrace has the widest breadth of services, including remote connection for door lock/unlock, remote vehicle start, a feature that consumers have been asking for for a long time as evidenced by after-market industry,” Peterson said. “We’re also delivering diagnostics data to dealers and adding the ability to access certain apps on the head unit without the user tethering their phone — that’s the big new thing.”
Verizon is exhibiting mbrace at CTIA Super Mobility 2015 in Las Vegas this week. mbrace services include navigation, location apps, remote safety and security features, and advanced travel assistance.
Embedded connectivity is a more expensive alternative to the bring-your-own-device (BYOD) model. Despite the embedded vs. BYOD debate in the industry at-large, Peterson doesn’t see one way as better than the other.
“Part of it is always demographics. With Mercedes-Benz customers, while the majority are connected with smartphones, the simplicity of having to use a radio head unit as opposed to their phones will be the right experience to create for premium customers,” Peterson explained. “Other price sensitive brands will continue to see BYOD as the way to go.”
Peterson admits the decision to equip all vehicles with the service is in part designed to prime customers so that the technology becomes a must-have. But, he adds, that the connected car creates a relationship between the manufacturer and consumer that benefits both, particularly with regard to safety.
While Verizon is the telematics provider to six OEMs in the U.S., including Volkswagon and Hyundai, Peterson doesn’t take the credit for decisions like the one Mercedes made when extending mbrace’s reach.
“I would call it a partnership heavily influenced by automakers. At the end of the day, the automaker decides what equipment to put into their vehicles. We provide all service, but they very much protect what goes into their car.”
One feature of mbrace that Peterson calls “a big deal” is the ability to turn your vehicle into a Wi-Fi hotspot. It’s a feature that has gotten caught between customer demands and regulator concerns that those increased demands will lead to spectrum-sharing that hampers the progress of vehicle-to-vehicle (V2V) communications.
“We are quite bullish on Wi-Fi in the car. You’re already equipping the vehicle with a connected device that’s Wi-Fi capable,” Peterson said. “We’ve done considerable research, and it’s all come back very positive.”
Differential GNSS+INS for Land Vehicle Autonomous Navigation Qualification
By Gilles Boime, Emmanuel Sicsik-Paré and John Fischer
Land-vehicle autonomous navigation requires centimeter-level qualification tools to enable confidence build-up for delivery to open-road traffic insertion. External positioning sensors over a dedicated road section can be replaced with an embedded high-accuracy, highly responsive epoch-by-epoch differential GNSS receiver coupled with an inertial navigation system. The demonstrated absolute accuracy and mobility extends the potential test area and minimizes cost for multi-environment validation.
Cover courtesy of Mercedes.
Personal cars and commercial trucks are continuously improving the driver experience and safety thanks to integration of more significant and machine-assisted control systems. Advanced driver-assistance systems (ADAS) are now integrated in all luxury cars and moving into mainstream products. Technologies covered by ADAS are specific for each car integrator, but increasingly they include now involving more safety features, such as driver assistance and partial delegation to autonomous control for small maneuvers such as lane control. The generation of ADAS systems introduced in early 2015 on high-end models are engaging more intelligence from the control system such as:
Lane departure warning system
Speed assistance and control
Driver assistance and control
Autonomous emergency braking.
It is not only individual drivers who want this technology, but also governments that are getting involved to prevent accidents and minimize the economic impact associated with them. In the European Union, the general safety regulation 2009/661 was the first step to engage member-states to act as a regulator to mandate car safety improvements. The European Transport Safety Council, a non-profit private association, released in March 2015 a position paper titled “Revision of the General Safety Regulation 2009/661.” It promotes the introduction of lifesaving technologies like intelligent speed assistance, autonomous emergency technology including all speed and pedestrian detection, and lane-departure warning systems as the next step of regulation.
Car manufacturers are not far behind. They understand their customers’ expectation of minimized risk and enhanced driving experience. Telematics is also a path to convert a single vehicle into a fully intelligent, connected and entertainment object with an associated high value. So every car manufacturer is willing to be seen as a technology master.
Toyota, for example, plans to integrate collision-prevention technology in all its mainstream and luxury cars by 2017. The ADAS new generation focuses on radar-activated cruise control technology for the collision-prevention system. The control system maintains distance from a vehicle ahead and can stop the car if driver doesn’t react. The next step is to monitor driver attention with sensors like cameras focusing on the driver’s eyes, and the pressure of the hand on the steering wheel.
However, no fully driverless car is expected in the next 10 years. This technology is limited by legal issues and the lack of reliable nationwide mapping data.
Since the technology must be fully proven to prevent any lethal threat on the user and other drivers, most car and truck companies are working actively on qualifying driverless technology today. Nissan began testing driver-assist technology on open-road traffic in Japan in late 2013. It enables highly advanced systems such as lane-keeping, automatic lane change, automatic exit, automatic overtaking of slower or stopped vehicles, automatic deceleration during congestion on freeways, and automatic stopping at red lights. This is a step towards attaining fully automatic driving, targeted for 2020 by Nissan.
Some European manufacturers such as Daimler Benz are also early adopters. Daimler/Mercedes uses the Bertha Benz prototype car to test autonomous driving technologies. It merged multiple vision, radar and GPS sensor with digital map to monitor an open-road 100-kilometer trip in August 2013 (Figure 1).
Figure 1. Bertha Benz test car, left, running fully autonomous 103-kilometer trip in open road including 27 percent narrow urban roads. Right, networked sensor systems of the S 500 Intelligent Drive research vehicle.
All manufacturers are building driverless capability into their technology demonstration concept cars:
Mercedes with F 015 Luxury presented at the Consumer Electronic Show, early 2015;
Audi with Prologue, an extrapolation of test car RS7 concept equipped with SuperFast driverless pilot;
BMW’s electric i3 car is integrating ActiveAssist technology that enables portions of drive to be without any manual intervention, such as car parking and autonomous rally to a meeting point;
Google’s self-driving vehicle that conforms to California license requirements for driverless tests in open traffic;
Tesla model SD autonomous test car.
Although most market leaders agree that this is not a technology for mainstream production in the next few years, they all work very efficiently to master the technologies. It is a big challenge to integrate all the sensors and the navigation functions to autonomously and accurately position the vehicle on a map. The whole system must be certified to prevent any liability in case of a crash, a case that would engage the solution provider and the vehicle manufacturer.
A large part of the qualification task will benefit from simulations and integration testing platforms in realistic conditions. At the very least, a very robust final open-space validation test must take place. Car manufacturers/integrators are using private test facilities in open air to perform serious trials before proceeding to real traffic conditions. Renault uses a 10-square-kilometer facility in France (Figure 2) to perform private tests in a protected area.
Figure 2. Renault outdoor test center at Aubevoye, France.
New autonomous car drive tests have mandated equipment enabling measurement of the car’s position on the track with an extremely high precision and repeatability. There are two competing technologies to do this:
Install many location sensors on the test track;
Use a general absolute positioning system.
Here we focus on an absolute positioning system that is affordable, easy to install and low maintenance. It is based on two main assertions:
The autonomous pilot can position accurately on the test track;
The test track is accurately referenced to the absolute positioning system.
We focus more closely in this article on the first assertion; the second one can be covered with a specific calibration trial where equipment, as discussed further, can be used in quasi-static mode and experience consistent accuracy. Let us have a deeper look at the candidate position technologies to verify autonomous pilot accuracy.
Positioning Technologies
Many technologies have been proposed to obtain vehicle position on the course. However, they all must be compatible with a reliable mapping database. Given the lack of consistent road infrastructure equipment with alternative capabilities, GNSS positioning is the sole enabling method to fit to a map every place around the world. That is why driverless systems always include a GNSS sensor to help other data matching with the map. The versatility and low cost of GNSS positioning makes it a candidate for open-air validation as well.
Standalone Standard Positioning Service GPS. The SPS single-frequency GPS receivers are included in so many nomadic appliances today that they are a commodity. Since their introduction 20 years ago, their performance is well understood. Some trials were performed in different area profiles with satellite constellation position dilution of precision (PDOP) < 2. Worse results were obtained from deep urban canyons in downtown Seattle, Wash.
For every technology, the relevant performance for the test course is the lateral error to the expected center of the lane in the two horizontal dimensions, referred to as 2D or N/E for orientation north and east.
For standalone SPS GPS, the lateral error standard deviation in 2D can be as high as 46 meters and have peak errors up to 660 meters. Lateral error in 3D can be as high as 20 meters with peak errors up to 175 meters.
Such performances are out of range for any positioning verification. It can only deliver a rough estimate of the point on the map, but would not provide tight correlation with other sensors for the navigation system.
Hybridized IMU and SPS GPS. Coupling of an absolute navigation GPS receiver with an inertial measurement unit (IMU) can mitigate corruption of the navigation solution when intermittent GPS signal outage is encountered. The hybrid approach is beneficial on any difficult signal transmission path from the satellite that is not line-of-sight: in urban canyons, deep foliage, under bridges, tunnels and in any multipath area. It also yields benefits in the very short term (less than a few seconds) for dispersion on the position computed from the sky.
Over the last 10 years, the combined benefits of micro-electro-mechanical sensors (MEMS) and tight coupling algorithms have raised the bar of positioning accuracy. It enables smoothed position along track and dead reckoning (DR) in case of GNSS signal outage.
Lateral error standard deviation in 2D is lowered to 2.3 meters and peak error up to 10 meters. However, this performance is still too poor to validate a vehicle position in the lane.
Hybrid Differential Single Frequency and IMU. The next step to mitigate systematic errors of the GNSS system is to use a set of multiple reference receivers in the vicinity of the area covering the test course. The reference receivers are static. The position of the reference is determined using long-term averages to mitigate constellation errors. A minimum for a position fix of 20 minutes is commonly reported. Then the position error standard deviation in 2D is less than 2 centimeters for baselines shorter than 100 kilometers.
For a MEMS integrated with a standard SPS GPS single-frequency receiver with DGPS correction on a mobile platform moving at less than 70 km/hour with HDOP < 1.4, Table 1 compares performance in a 2013 test.
Hybrid Differential Dual-Frequency Carrier Phase and IMU. The GNSS solution can be further improved, taking into account both L1 and L2 frequencies to mitigate propagation error and carrier phase to achieve ultimate signal accuracy. The combination of both helps solve ambiguities associated with the carrier-phase technique. When combined with a MEMS IMU, accuracy confirmed with HDOP < 1.6 is:
Lateral error standard deviation down to 0.18 meters;
Peak error of 0.6 meter.
However, this is still insufficient accuracy when compared to 0.1 meter required for verification testing.
With such low-cost IMU, GPS outages produce a rapidly increasing lateral error over elapsed time. The lower the speed, the poorer the position result.
Another limitation common to many differential solutions is the turn-on delay for the solution. It is also a repetitive issue in case of disruption of the GNSS solution. It extends the delay to recover from DR situation.
Geodetics’ Epoch-by-Epoch
Geodetics Inc. has developed a new class of instantaneous, real-time precise GPS positioning and navigation algorithms, referred to as Epoch-by-Epoch (EBE) and employing hybridized dual-frequency differential GPS with a high-performance IMU.
Compared to conventional real-time kinematic (RTK), integer-cycle phase ambiguities are independently estimated for each and every observation epoch. Therefore, complications due to cycle slips, receiver loss-of-lock, power and communications outages, and constellation changes are minimized. There is no need for the initialization period (several seconds to several minutes) required by conventional RTK methods.
More importantly, there is no need for re-initialization immediately following loss-of-lock problems such as those that occur when a mobile GPS receiver passes under a bridge or other obstruction, or when it loses satellite visibility during a shaded portion of road. In addition, EBE provides precise positioning estimates over longer reference-receiver-to-user-receiver baselines than conventional RTK.
This feature supports testing for long-range operations, for example, such as positioning a vehicle on a lane. The reference receiver is set in the vicinity of the test center track.
EBE requires the use of a minimum of two receivers, each of which is tracking a common set of five or more satellites and providing simultaneous dual-frequency phase data. Typically, one of the receivers is stationary, but this is not a requirement.
EBE has been proven utilizing dual-frequency receivers and operating at distances of up to 50 kilometers from the nearest base station in unaided mode. Additionally, the EBE algorithms operate in a network environment and make optimal use of all GPS measurement data at each epoch, gracefully degrading the position accuracies when some measurement data are not available. Furthermore, the system will make use of an IMU system, compensating for outages when line-of-sight to the satellites is blocked. This produces a robust and more reliable system.
Epoch-by-Epoch can deliver several benefits including:
Computationally efficient algorithms that provide a position estimate based on a single epoch in several milliseconds. This allows the real-time position estimate to be computed on the user platform (assuming reference station data is sent to the user platform).
An initialization period is not required. Since RTK requires some period of time (that can be measured in seconds to minutes) to perform ambiguity resolution, this is an important capability for platforms that:
require high accuracy (for example, for end-game scoring);
cannot see the satellites until launch;
have short flight or test course duration;
A re-initialization period following loss-of-lock is not required, unlike RTK, which needs to restart the integer-cycle phase ambiguity resolution process. This is another important capability because vehicle monitoring is considering EBE for dynamic applications where loss-of-lock and loss-of-data are likely.
However, it must be mentioned that many of the GPS receivers in use by the test (and training) community today do not support this dual-frequency requirement. Hence, those systems could not realize the maximum benefit.
This technology is implemented in a rugged modular platform (Figure 3) with three main units:
A dual-frequency GPS antenna,
An integrated INS coupling GPS receiver with either an internal MEMS IMU or external IMU,
An external fiber-optic gyroscope (FOG) IMU for high-end accuracy and reliability. The external IMU is optional and dedicated to increasing the DR capability.
Figure 3. Dual-frequency differential navigation unit hybridized with external fiber-optic gyro.
Performance. Tests have been performed in conditions close to the land-vehicle navigation validation. It is based on measurements on-the-fly with no post-processing except for evaluation of the error.
The first case is a static position of the rover 4.8 kilometers away from the reference receiver. Positions are updated once per second. The system includes a FOG IMU. the lateral error peak is less than 4 centimeters. Bias error is less than 1 centimeter. See Figure 4.
Figure 4. Single point error when rover is static.
The second test case is with a high-dynamic mobile platform, moving at a speed of 200 km/h, with an average distance from the reference to the rover of 6 kilometers. Lateral error standard deviation is 0.5 centimeters, peak error is less than 2.2 centimeters. Bias error is lower than 0.2 centimeters (Figure 5).
Figure 5. Dynamic trial test single point error.
The performance in these test cases meets the expected accuracy for validation of autonomous navigation.
One last method to increase accuracy is to switch to a different class of IMU performance, from tactical grade to advanced. When in the line-of-sight of the GNSS sky-view, the performance is the nearly the same.
Conclusion
A real-time, differential Epoch-by-Epoch, dual-frequency carrier-phase GPS receiver, tightly hybridized with a high-performance IMU can provide absolute error lower than 5 centimeters in the 10-kilometer baseline range of the reference static receiver. This is fully adapted to the qualification of driverless auto-pilot systems for the targeted year of 2020. It can avoid the need to use complex theodolite and vision calibration systems. It provides maximum flexibility and minimum sustaining costs.
Acknowledgment
This study has been made possible thanks to materials provided by Geodetics Inc. and the advice of Jeffrey A. Fayman, vice president, Business & Product Development, Geodetics Inc. The results displayed in Figures 4 and 5 are from a test with a medium-sized UAV from Allied Drones, model EF44 high-endurance quad.
Manufacturers
The Geo-iNAV family is a range of GPS-aided INS solutions available in different configurations, including various GPS receivers (L1, L1/L2 RTK, SAASM), internal MEMS or external FOG IMU. As part of this family, the Geo-RelNAV provides differential GPS relative navigation capability, the Geo-hNAV includes a dual GPS antenna receiver for static heading measurement capability, and the Geo-PNT combines position and attitude measurement with precise timing distribution.
Gilles Boime is is chief scientist for Spectracom. He is involved in GNSS signal generator, hybridized navigation platforms, GNSS timing and synchronization innovative solutions build-up. He holds an engineering diploma in telecommunication from Institut Superieur d’Electronique de Paris.
Emmanuel Sicsik-Pare is strategic product manager for Spectracom. He is involved in timing and navigation products and systems definition and application market monitoring. He holds a M.Sc degree from Telecom Bretagne.
John Fischer is CTO of Spectracom. He has more than 30 years experience creating navigation and communications systems, received his master’s in electrical engineering from SUNY at Buffalo. Prior to joining Spectracom, he worked in radar, command and control, and wireless systems.
Driverless cars are now traveling Britain’s public roads, according to a report in the Providence Journal. Four prototype self-driving cars took to the country’s highways, launching Britain’s first public trials. Still, with regulatory and legal hurdles, officials said fully driverless cars are unlikely to be used on British roads until 2030.
Four types of autonomous vehicles are being tested, including a shuttle that looks like a larger golf cart and a compact two-seater “pod.” Journalists took rides on the shuttle, which traveled a public square outside central London’s O2 Arena.
The project was “still in the early days,” Transport Minister Claire Perry told the Associated Press, but she added the new technology has the potential to make roads safer and attract global investment.
Driverless cars are also being tested in U.S. cities by companies including Google, and Apple and Sony are at least exploring autonomous vehicles. Meanwhile, automakers such as Audi, Mercedes-Benz and Nissan are developing autonomous cars, and Volvo expects to test its self-driving cars on customers by 2017. The widely discussed goal is to have autonomous vehicles on the market by 202o.
ABI Research forecasts that the global market for Driver Monitoring Systems (DMS) will reach 64.8 million units by the end of 2020 with the majority of shipments being accounted for in vehicles sold in the Asia-Pacific region. These findings are part of ABI Research’s Intelligent Transportation Systems Research Service and includes detailed installed base and forecasts of ADAS systems [advanced driver assistance systems] by regions.
Driver Monitoring Systems were first introduced as far back as 2006 when Toyota launched its innovative Driver Attention Monitor system. Toyota’s system functions by directly monitoring the driver’s face using a discrete in-dash camera and was initially offered as an option in the company’s luxury Lexus models. Other OEMs soon followed suit and announced their own DMS systems which were typically based on monitoring the vehicle rather than the driver’s face.
“DMS systems such as Mercedes-Benz’s ’Attention Assist’ and Volvo and Volkswagen’s ’Driver Alert’ systems were the first ADAS systems to be offered as standard equipment by OEMs, albeit only in a small selection of models,” comments Gareth Owen, principal analyst at ABI Research.
Today, an increasing number of ADAS systems are gradually becoming standard equipment in new cars, particularly in some European and Japanese brands such as Volvo, Mercedes-Benz, Nissan Infiniti, Lexus, and Mazda, and more are being offered as options. Although some of the big U.S. brands offer ADAS features in their European models, they typically do not offer the same features in their U.S. models, although this is beginning to change. Ford is a good example of this with its Ford Focus model.
“Another very observable trend in 2013 is that ADAS features are migrating from the luxury brands into B, C, and even A segment cars. Typically, the focus here is on offering ADAS systems, mostly as options, designed specifically for low-speed urban driving,” adds Owen.
Prices are decreasing, too. For example, the European Ford Focus offers an emergency braking system plus lane departure warning and lane-keep assist, driver alert, and blind spot monitoring as an optional package for £550 ($880) in the UK. Meanwhile, Volkswagen offers its City Emergency Braking System for £225-£405 ($360-$648), depending on model, on its budget A segment Up! car. This uses a laser sensor to detect the risk of an imminent collision and is active at speeds under 30 km/hr (18 mph).
