Tag: autonomous vehicles

  • U-blox updates ZED-F9R positioning module for slow-moving vehicles

    U-blox updates ZED-F9R positioning module for slow-moving vehicles

    U-blox has released a firmware update for its ZED-F9R positioning module, bringing robust centimeter-level positioning to slow-moving use cases such as robotic lawnmowers, e-scooters and shared bicycles.

    Based on the ZED-F9R high-precision sensor-fusion module, the u-blox ZED-F9R-02B was designed for use in autonomous and industrial applications that require simple and efficient implementation and where rapid access to highly accurate positioning data is key, even in challenging signal environments such as dense cities.

    The ZED-F9R uses the u-blox F9 multi-band GNSS receiver platform to concurrently track up to four GNSS constellations. Applying real-time kinematic (RTK) techniques, the module can derive positioning data with centimeter-level accuracy.

    The high-precision GNSS measurements are fused with data from the module’s inertial measurement unit (IMU), GNSS correction data, odometry information, and a vehicle dynamics model to provide high-quality positioning accuracy in situations where GNSS alone would fail. The receiver includes dynamic models for robotic lawnmowers, e-scooters and automobiles.

    The ZED-F9R-02B integrates the u-blox GNSS correction data service, broadcast in the highly scalable state-space representation format, making it easier to deliver fast time-to-market precise positioning solutions.

    An autonomous lawn mower trims the grass in a park in Finland. (Photo: Scharfsinn86/iStock/Getty Images Plus/Getty Images)
    An autonomous lawn mower trims the grass in a park in Finland. (Photo: Scharfsinn86/iStock/Getty Images Plus/Getty Images)

    Robotic lawnmowers achieve further improvements in terms of their positioning accuracy from the module’s ability to calibrate its dead-reckoning system even at low speeds. The module’s highly accurate position output lets end users precisely delimit the mowing area using software rather than with boundary wires, and deploy robotic lawnmowers on complex lawn shapes while excluding trees and shrubs. Users can align each pass of the mower with the next more accurately than before, delivering a professional-looking striped lawn.

    According to U-blox, combining these features allows product designers to use the module to create mowing equipment that enables “robotic mowing as a service” capable of operating in complex environments without the upfront cost and commitment of installing boundary wires.

    “Our new ZED-F9R-02B high-precision module is a turnkey dead-reckoning solution that will enable designers to bring products that need dead-reckoning technology to market more quickly than if they had to implement it from scratch,” said Alex Ngi, product manager, Industrial Navigation and Robotics, Product Center Positioning at u-blox. “It will also allow companies working in niche markets to employ dead-reckoning technologies without needing to build and maintain expertise in the subject.”

    The ZED-F9R-02B offers support for a range of correction services, including the SPARTN format, which will make it straightforward to distribute GNSS correction data over large geographical areas.

    iGPSPort cycling computer. In another announcement, U-blox and iGPSPORT, a sports hardware and data service provider operating out of Wuhan, China, have announced the iGS320 cycling computer, which builds on the u-blox M10 positioning platform. The cycling computer leverages the U-blox M10 platform’s ultra-low power consumption, compact size, and its ability to track all four GNSS to offer endurance athletes a superior user experience.

    Adopting the U-blox M10 positioning platform in its iGS320 cycling computer brought iGPSPORT an 80 percent increase the power autonomy to deliver 72 hours of continuous tracking, up from 40 hours in their previous solution. Concurrent reception of up to four GNSS constellations increases coverage and accuracy in otherwise challenging signal environments.

  • Skyward collaborates with Pix4D

    Skyward collaborates with Pix4D

    Screenshot: Skyward
    Screenshot: Skyward

    Skyward, a Verizon company, has announced its integration with Pix4D, a photogrammetry software suite for drone mapping. The partnership gives customers the ability to turn drone data into 2D maps and 3D models.

    Enterprises and drone pilots can now plan flights, receive FAA approval to fly in controlled airspace (LAANC), fly with Skyward’s InFlight ground control station, and process data using Pix4D from within the Skyward platform.

    Skyward Mapping & Modeling, powered by Pix4D, enables customers to create, view, measure, and export 2D orthomosaic maps and 3D photogrammetric models right from Skyward’s web app. With the processing power of Pix4D, Skyward customers can get business-ready data sets through a seamless plan, fly, process workflow.

  • DJI drones cleared for government use, plus more UAV news

    DJI drones cleared for government use, plus more UAV news

    There is a lot of UAV action taking place this month. An anti-drone defensive system is being trialed by the U.S. Department of Homeland Security (DHS). DJI drones get the green light again for use by government agencies. The U.S. Navy/Boeing refuel an F/A-18 aircraft using a drone, and the Mars helicopter drone has a non-fatal in-flight problem.

    DJI drones cleared for government use

    There’s good news for operators who might have been scared away from using drones manufactured by Chinese company DJI. Two “government” models have now been cleared by the Pentagon for use by agencies who had previously been instructed to cease and desist operating them. The Matrice 600 Pro and the Mavic Pro government models were cleared, because “no malicious code or intent” was found. These types of drone are now “recommended for use by government entities and forces working with U.S. services.”

    DJI Matrice 600 Pro (Photo: DJI)
    DJI Matrice 600 Pro (Photo: DJI)

    The drones have been modified from standard models to provide a customized control application, prevent automatic software updates being applied without user approval, and implement an operational control-zone capability.

    The U.S. Interior Department was behind the modifications, with the objective of resuming use of its grounded fleet of more than 500 DJI drones. They may need them again soon for forest-fire monitoring — it’s the season again. Allegations of potential “sharing of data” with the Chinese government were previously investigated and apparently debunked by Booz Allen Hamilton, in a report released in June 2020. Now the Pentagon seems to affirm these earlier findings.

    Nevertheless, DJI remains on the Commerce Department’s don’t-buy list, and the Justice Department has banned agencies from buying foreign drones that could be vulnerable to tampering with data, or otherwise might be a threat to the U.S.

    DHS trials anti-drone system

    In recent “bad-drone” incidents, people have tried to smuggle contraband into prisons, flown into landing flight paths at airports, and even successfully inflicted significant damage on oil storage facilities in Saudi Arabia — never mind a failed assassination attempt in Venezuela. And with illegal immigration and drug smuggling becoming crisis issues, DHS has decided that its probably time to build a level of protection at U.S. borders. A trial is now scheduled at the quieter and more orderly Canadian border so that DHS can fully focus on evaluating the Fortem SkyDome system.

    TrueView Radar (Photo: Fortem Technologies)
    TrueView radar detects and calculates the location and trajectory of hundreds of airborne objects quickly and accurately. (Photo: Fortem Technologies)

    Bad actors attacking a facility using a drone, sometimes with explosives strapped to it, have set their target as a waypoint and switched off the radio control link. The drone then carries on to its target, without fear of radio “sniffer” detection systems picking it up and jamming the control link.

    Fortem built a drone defense system that uses radar for “RF silent drone” detection. Not only does the Fortem SkyDome system detect unwanted intruder drones, but it also can send out a DroneHunter UAV which hones in on and fires a net to capture, immobilize and bring back the offending drone so the defenders can investigate the attack.

    Boeing UAV refuels F-18 fighter aircraft

    Boeing is engaged in development of an unmanned tanker vehicle for the U.S. Navy. The company has built and flown the first test drone, known as Stingray MQ-25 T1. Boeing has another seven unmanned refueling tankers on order. T1 is the first test article for the program. Following a number of successful initial handling, control and envelope extension test flights — including several with the Navy refueling pod installed under the T1 left wing — fuel was loaded, and on June 4, a Navy F/A-18 Super Hornet fighter aircraft accompanied the T1 into the air on another test flight.

    T1 Stingray refuels a Navy F/A-18. (Photo: U.S. Navy/Boeing)
    T1 Stingray refuels a Navy F/A-18. (Photo:
    U.S. Navy/Boeing)

    The F/A-18 flew close to and maneuvered around the drone for some time, and when the pilot and test managers concluded that both were in joint, controlled, stable flight, the MQ-25 was commanded to extend the refueling pod hose and drogue. The F/A-18 normally refuels in the air from another F/A-18 configured with the same refueling pod and refueling boom, so the actual transfer of fuel which then took place was somewhat commonplace for the F-18 pilot, save that the slipstream buffet was probably less significant. Nevertheless, this was the very first time that a Navy aircraft has been refueled by an unmanned aircraft.

    After further MQ-25 T1 test flights, the tanker-drone will be transported to Norfolk, Virginia, for deck handling trials aboard a U.S. Navy carrier later in the year. The MQ-25 Stingray is planned to take over this airborne refueling work which is currently handled by other F/A-18s, freeing up the fighter aircraft for operational fighter missions.

    Mars drone flight experiences navigation anomaly

    On May 25, the Mars drone Integrity had been commanded to take its sixth longer, higher flight. Like a good little tech package on the end of a 220-million-mile distant signal, Integrity did as it was told. However, after around 490 feet in the Mars atmosphere, Integrity started to change speed and dramatically oscillate back and forth. Even so, the UAV managed to complete the transit to within about 16 feet of the new intended parking spot.

    NASA analyzed the data sent back from the drone through the Perseverance rover. NASA discovered that the flow of images from the navigation camera had been disrupted by a missing image, and the time-stamps for the images had become incorrect.

    Integrity sits safely on Mars following an in-flight anomaly. (Photo: NASA)
    Integrity sits safely on Mars following an in-flight anomaly. (Photo: NASA)

    Integrity’s navigation system is fed by an inertial measurement unit with images from the nav camera, so the time-stamp discrepancies between what the camera was saying and what the IMU was saying gave it an unforeseen issue to resolve. The resultant bucking in flight was an effort to resolve the data time differences. Images are not used in the landing phase, so the oscillations ceased as the rover descended, and the landing was as graceful as usual.

    Fortunately, it appears that Integrity has not been damaged, and NASA is working to send the little drone on further scouting missions for its companion rover. NASA’s website indicates that the drone should have flown again on June 6, but right now there doesn’t seem to be any further evidence that a flight did or did not take place.

    Summary

    The testing that the Pentagon has done on a couple of models of DJI drones appears to have cleared them of possible malfeasance, but otherwise the report is classified. I wonder what secret science DOD used to detect/eliminate possible illicit communications with China by the drones? And it’s high time everyone realized that these “drone -thingies” can also be used to hurt us — as new technology in the hands of people who do not wish us well almost always can.

    This month I discovered that Navy F/A-18s could fly further by hanging refueling pods on other F/A-18s, thereby taking the tanking aircraft out of the regular training and combat rotation. I’m glad we’ve got a drone alternative coming that must cost a lot less and also will free up fighting assets.

    Finally, it doesn’t seem that we have to worry the Mars helicopter is too broken to keep seeking the best route for the rover. There were enough margins in the navigation algorithms that the little guy didn’t wreck when the image processing hiccupped. Let’s get flying again, NASA!

    Tony Murfin
    GNSS Aerospace

  • Sony launches Airpeak S1, its first professional drone

    Sony launches Airpeak S1, its first professional drone

    Sony’s first professional drone is small, but can be equipped with alpha mirrorless camera

    Sony Electronics Inc. has announced its first professional drone, the Airpeak S1. An introductory model in the new Airpeak line, the S1 is a small drone that can be equipped with Sony’s full-size mirrorless interchangeable-lens Alpha camera.

    The new drone uses a proprietary motor, propeller, control system and sensing technology, allowing it to fly at extremely high speeds with stable wind resistance.

    The Airpeak S1 features an advanced remote controller that can support the production of high-quality aerial images and freely control the aircraft. It also includes obstacle detection, automatic flight control via sensing and increased safety via cloud management of the aircraft and flight information.

    “Sony is excited to launch our new drone business with the Airpeak S1,” said Yang Cheng, Vice President, Imaging Solutions, Sony Electronics Inc. “Combining an extremely compact size with some of Sony’s most advanced imaging, sensing, AI and robotics technologies, the S1 will allow content creators, storytellers and commercial professionals to capture that which they’ve never been able to capture before.”

    Advanced flight performance

    The new Airpeak S1 offers dynamic flying capabilities, including a maximum speed of 55mph (90km/h), a maximum angular velocity of 180°/s, and a maximum tilt angle of 55°.

    Propulsion technology using a combination of key devices developed by Sony provides wind resistance in strong wind speeds up to 44.7 mph (20 m/s).

    In addition to the lightweight, highly efficient, sturdy and responsive proprietary 17″ propeller and brushless motor, the Airpeak S1 is equipped with an electric speed controller (ESC) for optimal control of these components. A unique, high-performance flight control system integrates the propulsion device and all sensor information to ensure stable flight and high maneuverability, Sony stated.

    Stable flight by sensing

    Stereo cameras equipped with Sony’s image sensors are installed in five locations (front, back, left, right, bottom) of the aircraft. Sony’s vision-sensing processor, which processes camera data at high speed and with low power consumption, and proprietary algorithms are used to accurately estimate the aircraft’s spatial position and orientation in real time, enabling stable flight even in environments where it may be hindered, such as indoors or under bridges.

    The Airpeak S1 is also equipped with a unique high-performance flight-control system that integrates all sensor information such as inertial measurement unit (IMU), direction, barometric pressure and infrared ranging to optimize the propulsion device.

    The Airpeak S1 uses multi-directional sensors to enable its obstacle braking function. The front, rear, left and right stereo cameras and the infrared range-finding sensor mounted on the top recognize obstacles in the vicinity of the aircraft, allowing the aircraft to automatically decelerate and stop according to its behavior and the surrounding conditions.

    Photo: Sony
    Photo: Sony

    Camera and lens variations

    Users have the flexibility to choose the right Alpha system for their desired application to expand their filmmaking horizons. The Airpeak S1 is compatible with a wide range of camera bodies including:

    • Alpha 7S series and FX3 for high descriptive capability with suppressed noise
    • Alpha 7R series for ultra-high definition
    • Alpha 9 series and others for distortion-free images

    The Alpha 1, which can shoot footage in 8K, is also compatible.

    The drone includes Airpeak Flight, an iOS- and iPadOS-compatible application that integrates the aircraft, transmitter, camera and gimbal, allowing the operator to monitor status information such as flight distance and remaining battery power, and change various operations and settings on the screen.

    Airpeak S1 also includes a dual-operation mode so that one user can operate the drone while another user can operate the gimbal and camera simultaneously while checking the same image, even in complicated scenes. The first-person view camera, which can be tilt-operated from the remote controller, is mounted on the nose of the aircraft and is useful for the operator to check the direction of the aircraft and the direction of travel.

    Photo: Sony
    Photo: Sony

    Airspeak Base

    The new drone includes “Airpeak Base,” a web application that allows the operator to manage equipment, create flight plans, and manage flight logs. In the equipment management, information on the equipment used is automatically listed and managed based on the flight log. This allows the operator to check the condition of the aircraft before going to the field, minimizing on-set issues.

    With Airpeak Base, the operator can create advanced flight plans and automatically fly the aircraft along the same course repeatedly, as if the drone were on rails installed in the air. It is able to set the position (latitude, longitude and altitude) and speed of the aircraft along the timeline, and specify the orientation of the gimbal and the timing of video or still image shooting. It can also draw smooth curves on the map. Reproduction flight is an automatic flight function that reproduces the flight route, gimbal, and camera movements based on the flight logs that have been flown in the past.

    All aircraft information, including logs, can be uploaded to the cloud via the mobile app Airpeak Flight. Airpeak Base allows users to check the status of the aircraft and can provide notifications if necessary. Users can manage the status of all managed aircraft in one place before flight. Past flight logs can be viewed on the logbook screen, and details such as in-flight errors can also be reviewed.


    Sony Group will be exhibiting Airpeak S1 at Japan Drone 2021, a drone exhibition to be held at Makuhari Messe in Chiba, Japan, June 14-16. In addition, Izumi Kawanishi, Senior Vice President, AI Robotics Business, Sony Group Corp. will provide the keynote speech on the morning of June 14.

  • UAV Navigation autopilot powers Power4Flight engines

    UAV Navigation autopilot powers Power4Flight engines

    UAV Nav logoUAV Navigation’s range of Vector-autopilots is now integrated with Power4Flight’s IntelliJect EFI, designed for use in small-engine aerospace applications.

    Using the robust and extensive communication capabilities of UAV Navigation autopilots, the IntelliJect EFI’s engine control unit can communicate and deliver critical parameters through the CAN port. This way, the autopilot is able to automatically control and monitor the engine. The operator will be capable of controlling and receiving real-time status information from the engine in the UAV Navigation’s advanced ground-control station Visionair.

    The IntelliJect EFI is highly configurable for a variety of engine types (two-stroke, four-stroke, triples, twins and singles), including Power4Flight’s engines or any other manufacturer engine.

    Power4Flight propulsion systems and electronics are used in a wide range of unmanned aircraft systems. With this integration, the fuel injection systems of the American manufacturer will be able to send performance parameters, such as RPM, temperatures, pressures, throttle range and error to the autopilot, and receive commands.

  • Commercial UAV Expo Europe delayed to January due to COVID-19

    Commercial UAV Expo Europe delayed to January due to COVID-19

    Commercial UAV Expo logoOrganizers of Commercial UAV Expo Europe have announced that the next edition of the event will take place with Amsterdam Drone Week (ADW) and EASA High Level Conference at RAI Amsterdam Convention Center, Jan. 18-20, 2022.

    Previously, the events were scheduled to take place together in December 2021, but the organizers have decided — in close consultation with partners Amsterdam Drone Week and EASA — to delay the event until early 2022 due to the impact of COVID-19 on travel and the event industry.

    The new date will offer partners, speakers, visitors and delegates more certainty to travel to Amsterdam and thus bring the worldwide UAM/UAS community together. Future editions will revert to the usual December time frame.

    “The benefit of co-locating the three events is in bringing the whole drone ecosystem together in one place where professionals can learn, connect and drive the industry forward,” said Lee Corkhill, group event director at Diversified Communications, organizer of Commercial UAV Expo Europe.  “Commercial UAV Expo Europe is the leading international trade show and conference in Europe for the commercial drone market delivering sUAS manufacturers and end-users. Amsterdam Drone Week focuses on Urban Air Mobility and the role of cities. The EASA High Level Conference focuses on the stakeholders involved in UTM and regulatory issues.”

    During ADW 2021, the EASA High Level Conference will take place on Jan. 18-19 and the Commercial UAV Expo Europe will be held on Jan. 19-20.

    ADW will offer a hybrid option in January 2022, meaning content from the ADW mainstage will be broadcast daily, including matchmaking and online speed networking sessions.

    In 2020, ADW organized a successful online event. “Although it was a great success given the circumstances, we prefer to see the fast-growing industry around drones and unmanned aircraft back in Amsterdam in person,” said Paul Riemens, CEO RAI Amsterdam. “Bringing people together is part of the DNA of RAI Amsterdam.”

    “Innovative Air Mobility is an area of intense activity for EASA. We are steadily putting all the rules in place for these operations and working on certification for these entirely new types of aircraft,” said EASA Executive Director Patrick Ky. “The High Level Conference at ADW is an important platform for discussions with a range of stakeholders on these developments.”

    Commercial UAV Expo Europe, taking place as part of Amsterdam Drone Week along with the EASA High Level Conference, is a leading international trade fair and conference in Europe focusing on the integration and operation of commercial UAS. Industries covered include

    • Construction
    • Drone delivery
    • Energy and utilities
    • Forestry and agriculture
    • Infrastructure and transportation
    • Public safety and emergency services
    • Security
    • Surveying and mapping

    The next edition will take place 18-20, 2022 at RAI Amsterdam.

    Commercial UAV Expo Europe is presented by Commercial UAV News and organized by Diversified Communications, organizer of Commercial UAV Expo Americas, Geo Week, and Geo Business Show. For information about exhibiting at Commercial UAV Expo contact Katherine Dow, Sales Manager, at [email protected] or +1.207-842-5497. For attending information, visit www.expouav.com or email [email protected].

    EASA High Level Conference

    The main theme of the EASA High Level Conference will be “UAM Becoming a Reality.” Main topics are:

    • Multi-Level Governance of Urban Skies
    • Integration of new entrants in Urban Airspace and overall U-space-ATM integration
    • Mobility Hubs: drone and eVTOL landing sites and vertiport.

    The outcome of EASA Social Acceptance study and status of the implementation of the key findings will be discussed. There will be presentations and discussions on concrete implementation projects.

    In addition, educational and technical sessions, workshops and roundtables will be held on, among others, the application of the guidelines related to design verification of UAS, technical requirements on vertiports, counter drones technology and U-Space implementation.

    Amsterdam Drone Week

    Amsterdam Drone Week is the global platform for sharing knowledge on current air solutions, potential innovations, and vital regulations. Creating valuable connections with all key players, knowledge institutes and authorities to co-create and cooperate to build the UAM ecosystem. Through live and virtual summits, expert sessions and matchmaking, ADW connects the entire UAM value chain around drone regulations, new technology and future solutions.

     

  • US Congress may move against Chinese-made drones

    US Congress may move against Chinese-made drones

    The FLIR M440 UAV. (Photo: Teledyne FLIR)
    The FLIR M440 UAV. (Photo: Teledyne FLIR)

    A bill moving through the U.S. Congress would impose a five-year ban on United States government purchases of drones manufactured or assembled in China, reports The Associated Press. The measure reflects bipartisan concerns that the unmanned aerial vehicles (UAVs) made in China could facilitate Chinese spying on critical infrastructure.

    Major commercial and consumer UAV-maker company DJI is based in Shenzhen, China. Many of its small, low-altitude drones are employed by local and regional government users in law enforcement, emergency response and surveying. The ban could affect police departments that rely on federal funds for equipment. In 2020, the Department of Homeland Security halted such grants for Chinese-made drones.

    Chinese-made components, including GNSS receivers and inertial sensors, are not addressed in the bill, and the Pentagon has acknowledged that many components for non-Chinese-company drones are made in China.

    While the ban wouldn’t go into effect until 2023, many federal agencies have already imposed temporary restrictions on the use of Chinese drones. The Interior Department had flown more than 11,000 drone missions before January, when the agency temporarily grounded its fleet of more than 500 DJI drones over cybersecurity concerns, according to The Hill. The Hill cites a May 6 Pentagon report. The report found no malicious code in the software for DJI’s Government Edition drones.

    An analysis by Booz Allen Hamilton released in June 2020 found no evidence that DJI drones have shared sensitive information with the company or the Chinese Communist Party.

    In August 2020, the Defense Department issued approval to drones from five companies:

    • Skydio’s X2-D. Skydio is based in Redwood City, California.
    • Parrot’s Anafi USA. While Parrot is based in Paris, France, the ANAFI USA drone is manufactured in the United States for U.S. customers.
    • Teledyne FLIR’s Flir M440 Ion. The drone was originally made by Altavian in Florida, which was acquired by FLIR in December 2020, which was subsequently acquired by Teledyne Technologies in January. Teledyne FLIR is headquartered in Wilsonville, Oregon.
    • Teal Drones’ Golden Eagle. Teal Drones is based in Salt Lake City, Utah.
    • Vantage Robotics’ Vesper. Vantage is based in San Leandro, California.

  • GNSS in the fast lane: Meeting autonomous vehicle navigation challenges

    GNSS in the fast lane: Meeting autonomous vehicle navigation challenges

    Autonomous vehicles are being tested both on open roads and in controlled environments. (Photo: Trimble)
    Autonomous vehicles are being tested both on open roads and in controlled environments. (Photo: Trimble)

    The advent of autonomous vehicles (AVs) is one of three revolutions in the automotive industry that will likely change this country as much as cars did over the last century. The other two are the conversion from internal combustion engines to electric ones and the integration of cars into digital traffic networks.

    Once mass deployed, AVs promise to dramatically reduce the number of traffic fatalities (42,000 in the United States in 2020, a National Safety Council report shows). They will never be sleepy, distracted, aggressive or drunk — nor will they engage in such inane human driving behaviors as texting while driving, playing chicken with bicyclists, or running red lights. They also promise to reduce fuel consumption, harmful emissions and traffic congestion by optimizing routes and increasing the number of people using car services instead of owning their own car.

    To realize this vision, however, cars will have to do a lot more than just find their way on their own. They will have to perform flawlessly in an unpredictable world that includes toddlers, reckless drivers, fallen trees, sinkholes, construction and accidents.


    See also Testing autonomous vehicles inside and out


    GNSS Plus Corrections

    Among the many sensors aboard an AV, the GNSS receiver has a unique role. It is the only one that can provide absolute positioning, in the form of latitude and longitude coordinates, to within a couple of decimeters anywhere on Earth. As such, it is “a key enabler to a lot of the vehicles to know precisely where they are and whether it is safe to activate autonomous systems,” says Gordon Heidinger, automotive segment manager, Autonomy and Positioning division at Hexagon.

    A GNSS receiver cannot achieve the level of accuracy required for autonomous driving without robust corrections. Fifteen years ago, the state of the art was real-time kinematic (RTK) corrections. However, “the cost of that equipment exceeded the cost of a small car at that time,” recalled Steve Ruff, general manager, On-Road Autonomy Division at Trimble. “They were targeting a system cost of about $200. Today, that number is below $50, including the antenna, the GNSS positioning engine, and the software that runs on it.”

    Today, all automotive manufacturers are using a form of precise point positioning (PPP) corrections, which is a one-way broadcast, as opposed to the two-way communication between a base station and a rover required for RTK. This means that a single correction stream can serve an entire continent, Ruff pointed out. “Once a vehicle is manufactured, we will support it with our PPP corrections stream for at least 10 years, which is the typical service life of a vehicle.”


    Obstacles to Adoption

    To achieve mass-market adoption, AVs will have to overcome numerous and complex obstacles:

    • The technical difficulty of dealing with a limitless number of unanticipated challenges, such as poor visibility because of weather conditions, unpredictable human behaviors, complicated obstructions, detours and potholes
    • The need to map millions of miles of roads, develop vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications, and protect vehicle software from hackers
    • The difficulty, if not the impossibility, of handing off control to a human quickly enough to be safe when the system is unable to deal with a complex situation
    • Questions about legal responsibility and insurance liability
    • Ethical dilemmas about how to program the system to respond in emergencies
    • The development of appropriate federal and state regulations
    • Resistance from paid drivers who fear losing their jobs, including 3 million U.S. truckers, and from many other drivers, who fear losing control over their safety.

    Trimble has approached all the major car manufacturers, has several programs in development, and has received multiple positioning requests for information (RFIs), Ruff said. “In 2018, Trimble’s RTX corrections service was the first solution adopted for production use in passenger vehicles, providing absolute precise positioning for General Motors’ Super Cruise system.”

    Additionally, Trimble is working with Qualcomm and with SiriusXM, which will deliver Trimble’s RTX corrections over its satellite network, just like it does with music. “It is a good partnership because about 80% of the vehicles in North America are coming equipped with SiriusXM radio technology,” Ruff said. “The OEMs do not have to buy any additional hardware.” RTX corrections can also enter a vehicle via cellular IP, L-band satellite broadcasts and, potentially, over a V2I link.

    Hexagon has proposed a PPP solution for automotive, “mainly because we essentially have the world covered with base stations, and that is a hard thing to do,” Heidinger said. “We have been running a corrections network for a very long time.” PPP’s one-way broadcast offers better cybersecurity because the GNSS receiver does not have to disclose its position, he added.

    Swift Navigation is building a global corrections network. To make it suitable for the automotive market, the company is aiming to make its corrections service affordable and scalable. “We realized quickly that neither of the traditional RTK and PPP approaches were going to meet those requirements,” said Fergus Noble, company co-founder and CTO, “so we invested in developing a corrections service pretty much from the ground up.”

    RTK provides high accuracy and short convergence times but is typically costly to deploy because it requires a very high density of stations, Fergus explained. As a consequence, most providers do not have continuous coverage over a wide area. Conversely, while PPP is a true global solution, it is less accurate and takes a long time to converge. “That may be fine in a marine or land-surveying application, but not if you are driving through city tunnels and bridges and need it to be able to reacquire a high-accuracy position within a matter of seconds. Therefore, we took a hybrid approach, together with a lot of new IP that we developed.” The service provides coverage in all the United States and most of Europe, and is being tested in Japan, South Korea and Australia.

    Accuracy and Integrity

    A common target accuracy for lane-level positioning is 20 cm 95% of the time. That means that AVs need to know when their positioning accuracy falls beneath that threshold. “We are building into our positioning solutions an accuracy metric that is output along with the position information we are providing,” Ruff said. “[The metric] can be used by the intelligence in the system to decide whether it can rely on the GNSS solution or needs to switch to one of the other complementary technologies because GNSS accuracy is not fulfilling its lane discipline.”

    Heidinger noted the importance of economies of scale when mass-producing vehicles, where cost and ease of manufacturing become factors. “We can take some of our high-end equipment and get you 2 cm of accuracy with this technology, but the price point and the feasibility of this going into mass production for automotive is not favorable,” he said. “So, we’ve taken the approach of providing a software positioning engine that can be fit onto any hardware.”

    Hexagon is developing products in partnership with STMicroelectronics, using the company’s Teseo V family of measurement engines. “ST is one of the established leaders of automotive GNSS solutions,” Heidinger said. “We take their measurements and put our positioning and corrections solution behind that to give positioning with lane-level accuracy.”

    Noble agrees on the importance of knowing the reliability of a vehicle’s GNSS-based lane accuracy. The prevailing approach, which fuses data from GNSS and other sensors, makes it acceptable for one data source to be temporarily unavailable if the system is aware of that outage, he said. “That is where you start to see Swift, and others as well, focusing on the notion of integrity.”

    An AV’s level of autonomy determines its behavior during GNSS outages. For systems with Level 2 autonomy and below, the driver must remain engaged, while Level 2+ and Level 3 systems will alert the driver to retake control when needed. If a driver of a Level 2+ or higher system fails to reengage, the AV’s reaction depends on the system and manufacturer.

    “When we start to see Level 3 or above self-driving systems come onto the market, they will require that the GNSS component has an ISO 26262 safety certification,” Ruff said. “Many companies, including Trimble, are going through, or have gone through, the process of safety-certifying their offerings. As part of the AV system’s safety architecture, they will build in the capability to safely curb the vehicle if the system detects a malfunction or a spoof or some other type of problem.”


    Automation Levels

    In 2014, the international Society of Automotive Engineers released a standard, adopted in 2016 by the U.S. National Highway Traffic Safety Administration, that classifies cars in six levels, ranging from Level 0 (no automation) to Level 5 (full automation, meaning vehicles that can handle the full spectrum of road and traffic scenarios without any assistance from the driver). While many production models already incorporate various forms of Level 1 driver assistance, no current production car exceeds Level 2, or partial automation, which requires the driver to monitor the vehicle’s surroundings and take over as necessary. No test vehicle has yet achieved Level 5.
    Image: GPS World
    Image: GPS World

    Other Sensors

    Beyond lane-level positional accuracy, safe driving also requires avoiding collisions with other vehicles in the same lane or straying into it. Cameras, lidar and radar will detect other vehicles as well as fixed infrastructure and random obstacles, measure their distance, and monitor their movement.

    While lidar scanners are better than cameras as detecting sharp-edged features, such as curbs, cameras are better at detecting and interpreting visual cues, such as road signs and the location and curvature of lane markers. In bad weather, radar is essential, because radio waves, unlike light waves, can penetrate rain, snow, fog and even dust, enabling radar to “see” where cameras and lidar cannot. However, radar sensors cannot see much detail, and cameras do not perform well in conditions with low light or glare.

    Besides providing data about a vehicle’s trajectory, inertial navigation systems (INS) also measure its attitude (roll, pitch and yaw), enabling the software to better correlate and interpret data from the other sensors.

    For example, when a car brakes sharply, its front end goes down; any forward-facing sensors measure distances to points closer to the car than they did a moment earlier, when its chassis was parallel to the street surface.

    INS can also detect unsafe conditions, such as excessive slip angle, which is the angle between the direction of the rolling wheels and the vehicle’s true heading. A slip angle as small as 0.5 degrees can trigger skidding, spins or rollover, especially in the case of SUVs and tall trucks. Wheel-speed sensors also help verify the vehicle’s movement.

    “All these technologies have their limitations,” Ruff said. “However, if you design the system, including all these technologies, then you can come up with a robust, safe combination that will enable autonomous driving.”

    In addition to helping to avoid collisions, these other sensors provide relative positioning by comparing the images they acquire with highly precise maps to help locate the vehicle, especially in urban environments, which are well mapped and rich in recognizable landmarks.

    Imagine an AV moving through different environments. It might travel from a city with urban canyons that degrade GNSS navigation, yet with landmarks that help relative positioning, to a rural environment devoid of both. The AVs’ algorithms must constantly weigh how much to rely on the different sensors. “Many of the OEMs and car companies are seeing that even rain mist on a highway is very bad for lidar and cameras, because it creates a big blur, but that is where GNSS will perform really well because it is open sky,” Heidinger said. “So, the two types of sensor systems complement each other very well.”

    “Odometry sensors, such as a wheel-speed sensors, minimize any potential drift and add robustness to data that may have a GNSS outage of greater than 5 seconds, such as longer tunnels,” said Wesley Hulshof, principal engineer – ADAS Testing at Racelogic.

    Photo: Racelogic
    Photo: Racelogic

    Noble sees a split in the industry. Companies such as Waymo and Cruise are pursuing Level 5 autonomy and are “heavy users of lidar” as well as other sensors. Companies such as Swift are focusing on Level 2 and Level 3 series production vehicles. “If you are making a mass manufactured vehicle for the production market, it rules out using a lidar sensor,” Noble said. “It is just too costly and complex right now to use. So, typically, if you look at the systems that are out on the market today, such as a Tesla Autopilot or a GM Super Cruise, they are very reliant on the camera as the primary sensor. Obviously, also inertial and some use of radar.”

    Maps and Communications

    While accurate and up-to-date maps have an important role to play in making autonomous driving possible, the more detailed maps are, the more the world they describe is constantly changing.

    Meanwhile, the sensors keep improving and dropping in price, making maps less important. In the end, AVs — like human drivers — will probably rely much more on their ability to “see” and analyze their environment moment-to-moment.

    Also like their human counterparts, they will gain experience. Unlike human drivers, however,  AVs will be able to instantly share their experience with every other vehicle in their area via vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications.

    V2V communications will enhance safety by informing AVs of the trajectories of nearby vehicles. If a vehicle is speeding toward an intersection and not slowing for a red light, it will be communicating its position and trajectory to other cars over a V2V link, Ruff explained.

    “Then your car can make the intelligent decision to pump the brakes and avoid that collision. The same positioning stack that operates as part of the AV stack can also be used to support V2V-type applications, and the position of the vehicle will be much better than what the current V2V spec states.”

    Different Approaches

    Each GNSS manufacturer is taking a different approach to AV positioning.

    The worlds of traditional automotive positioning and the products on which NovAtel has historically focused are coming together, Heidinger said. “The autonomous technology is demanding it and pushing for higher performance and safety-of-life functionality. Hexagon is bringing high-performance positioning solutions to the automotive industry in a manner that accepts automotive manufacturability, quality and efficiency.”

    The company has also joined the 5G Automotive Association (5GAA), a large consortium developing AV solutions. “There are probably 100 companies in the industry coming together and helping to develop that vehicle-to-network communications solution, including telecom partners and automotive partners, and we are providing the GNSS expertise,” Heidinger said. “To meet the high-volume production-intent applications, including automotive quality, we recently developed a receiver based off the ST Teseo V family of measurement engines. We have an ST Teseo V set of chips on the PIM 222A product that launched in May geared exactly toward the automotive market.”

    By contrast, Trimble is not focused on providing GNSS receivers or other hardware. “We allow the Tier 1 automotive manufacturers to architect the system using the components that they have selected from their preferred suppliers,” Ruff said. “We tailor our positioning solution to work with their architecture. So, we are agnostic as to the selection of the GNSS receiver, the IMU, the operating system running on the host system, and the host processor that runs the software. We can adapt our stack to run on virtually any system, using measurements from any GNSS source that meets our API requirements.”

    For Swift, its “vision from day one has been to bring this type of precise positioning technology to mass market applications, such as automotive, which is a big focus for us,” Noble said. “That includes autonomy, but also ADAS, HD navigation and V2X. We do not want to be a hardware supplier in the automotive supply chain. Our boards are focused on professional and industrial markets.”

    Swift’s automotive software, called Starling, runs on the vehicle’s computer. To generate a precise position, it ingests raw sensor data, as well as corrections data from the company’s Skylark network. “We focus on providing a precise-positioning stack that layers on top of any of this current generation of low-cost, automotive-grade receiver hardware from companies like STMicroelectronics.”

    This test in London shows the value of inertial and wheel speed sensors. (Image: Racelogic)
    This test in London shows the value of inertial and wheel speed sensors. (Image: Racelogic)

    The Future

    Speculation abounds as to when AVs will enter mass production and how the transition from human to robotic drivers will take place. “There might be a ‘classics only’ lane in the future,” Heidinger said “that will be the only place where cars are allowed to be driven manually.”

    Safety-enhancing automotive devices typically start out as optional extras, then get incorporated into best-practice standards promoted by independent bodies. Eventually, they become compulsory.

    Some automakers have committed to creating their own AVs, while others are intent on creating a turnkey solution to transform conventional cars into driverless models. However, the initial market for AVs likely will be commercial fleets rather than individual consumers.

    “It will still take quite a few years before we see cars take over and drive themselves, because legislation, insurance and these sorts of things will have to happen along with the technological advances,” Heidinger said. “But the positioning side is becoming more defined. We are seeing things like L5, the Galileo constellation, coming in and becoming more available. There are more constellations providing more data for use in our solutions, so that is promising.”

    Swift’s Noble said, “Most of the major manufacturers working on Level 2+ and Level 3 systems are realizing that precision GNSS will be a key component of their architecture. Most of the major OEMs have signaled some level of intent to integrate this technology. Most are tracking to start the program next year,” he added.

    “We envision that in five or six years every vehicle will have a single positioning utility on board that will serve all the location-aware applications on the car — whether it is an autonomous vehicle, V2V or V2I,” Ruff said. “It will meet the most stringent accuracy requirements from all the applications and serve navigation, telematics, security, V2X and AV/ADAS applications.”

  • Testing autonomous vehicles inside and out

    Testing autonomous vehicles inside and out

    A test of Racelogic’s parking assistance system. (Photo: Racelogic)
    A test of Racelogic’s parking assistance system. (Photo: Racelogic)

    Racelogic helps vehicle manufacturers develop autonomous vehicle technology and test them on indoor test tracks and the open road.

    Racelogic helps vehicle manufacturers develop autonomous vehicle (AV) technology and testing houses test them. Over time, regulatory and consumer testing has evolved from indoor test tracks to outdoor open-road tests, and then to indoor controlled test environments.

    “Due to their application, advanced driver-assistance systems (ADAS) originated and are still mainly developed and assessed on open-sky, controlled test tracks, tackling the most common killed and seriously injured (KSI) accident types,” said Wesley Hulshof, principal engineer – ADAS Testing at Racelogic. “These assessments usually make use of sophisticated driving robots for closed loop, centimeter-accurate path following and precise speed-controlled test-track assessments. The robots can only attain this accuracy by being fed the speed and positional data by GNSS sensors, such as the Racelogic VBOX.”

    Racelogic’s VBOX GNSS receiver. (Photo: Racelogic)
    Racelogic’s VBOX GNSS receiver. (Photo: Racelogic)

    Accuracy is key to conducting assessments for the European New Car Assessment Programme (Euro NCAP) and the U.S. National Highway Traffic Safety Administration (NHTSA). Using GNSS in conjunction with RTK base stations provides centimeter-level accuracy in position, said Hulshof, as well as accurate speed and heading information to measure ADAS data to both static and moving targets. Additionally, combining a GNSS receiver with an inertial measurement unit (IMU) allows for low-drift, high-accuracy speed and positioning information within areas of high GNSS multipath or temporary occlusions, such as gantries, bridges, forests or built-up areas.

    However, “people do not just drive on closed test tracks with accurately positioned targets and infrastructure,” Hulshof said. “They do not drive at a constant throttle position and maintain an exact time-to-collision to the vehicle in front of them, like robots do. In fact, people often drive erratically.”

    For these reasons, testing houses are conducting supplementary assessments on the open road, under real-world conditions. In these conditions it is still important to know vehicles’ positions and speeds to localize them and validate the system’s sensors, networks and algorithms.


    Testing Stages

    Stage I: Controlled
    ADAS was developed for outdoor use because this is where car crashes occurred. For this, an open-sky GPS signal was essential for positioning. The types of tests and level of scientific rigor meant that the tests could be performed on closed test tracks.

    Stage II: Randomized
    Tests were brought to the open road to add elements not found within a closed environment such as traffic and higher speeds of the vehicle under test. For this, extra sensors were employed to add robustness in areas of obscured GNSS coverage.

    Stage III: Controlled
    Testing is brought back indoors for climate control and to assess L3/L4 AD functionalities such as valet parking.


    Because open-road testing does not permit being constantly within range of a static base station, Racelogic developed a moving base solution for open-road testing that gives accurate relative positioning between two or more vehicles.

    The increased demand for real-world testing of ADAS has generated demand for reliable ground truth data. “For example, if you consider a car driving on the winding roads of the Italian Alps and the position is out by 2m,” Hulshof said, “that is the difference between lovely scenery and falling off the side of a cliff. So, you need centimeter-level accuracy in the positional algorithms of the self-driving car, but also in the assessment tools, while we are testing it. For that reason, we still need GNSS and would ideally need RTK.”

    To meet this demand, Hulshof said, Racelogic produced its own networked transport of RTCM via Internet protocol (NTRIP) solution, consisting of a modem and associated service provider. It allows for global coverage of high-accuracy, absolute positioning of a test vehicle in open-road conditions. Both the NTRIP and the moving base solutions allow ADAS testing to centimeter-level accuracy on the open road without the need to be in radio range of an RTK base station, thereby greatly expanding the testing possibilities.

    “Whilst both the NTRIP and the moving base options allow for high-accuracy positioning,” Hulshof said, “they are still reliant on having an open sky for good GNSS coverage. IMU integration allows for improved accuracy over short periods of occlusion, but to truly give as accurate a signal as possible we need to be open to accept information from multiple satellite sources. That is why highest longevity accuracy is only achieved by using the GPS, GLONASS, Galileo and BeiDou constellations to provide the best RTK positioning performance in areas where that was not previously possible.”

    To control the environment and allow for year-round testing, test laboratories such as the Insurance Institute for Highway Safety (IIHS) facility in Arizona and Asta Zero in Sweden have purpose-built covered test facilities, giving shelter from extreme heat or cold. Testing inside both set-ups, however, still relies greatly on the test vehicle positioning. Standard positioning techniques via GNSS in these situations is simply not possible. Therefore, Hulshof said, Racelogic designed the VBOX Indoor Positioning System (VIPS), which allows for seamless testing indoors or outdoors. “Because this system works as an alternative to satellites, with the in-vehicle VBOX allowing RTK-level performance without GNSS, the test vehicle can travel from open-sky outdoor testing to a closed environment seamlessly, with no drop in data during the transition or afterward.”

    Finally, Hulshof said, ADAS and AD systems have moved on from straight-line highway scenarios to low speed turning scenarios often performed away from the open sky previously required for accurate GNSS coverage. Examples include multi-story parking garages and valet parking. “Scenarios such as self-parking and park-assist assessments, as well as indoor L1 ADAS, are becoming increasingly common requests by manufacturers on test facilities.”

    These environmentally controlled facilities can simulate real-life conditions that affect specific sensors — such as sensor flare, fog, mist and water films. These types of facilities use VIPS to give outdoor GNSS accuracy in an indoor controlled environment. “There is a trend toward bringing the testing from closed test track to randomized real world back into a highly contained, climate-controlled area,” Hulshof said. “We then have an option for anything.”

  • Thales Alenia Space to assess feasibility of EGNSS integrity service

    Thales Alenia Space to assess feasibility of EGNSS integrity service

    Image: loveguli/E+/Getty Images
    Image: loveguli/E+/Getty Images

    Thales Alenia Space, a joint venture between Thales (67%) and Leonardo (33%), has been selected by the European Commission for a new strategic contract to assess the feasibility of an integrity service to complement the European Global Navigation Satellite System (EGNSS) High Accuracy service, which will pave the way for use in autonomous vehicles.

    Thales Alenia Space will focus on the development of a sensor-fusion approach, including and complementing evolutions of EGNSS High Accuracy. These service evolutions are aimed at providing the integrity level to serve the high-reliability and high-accuracy positioning needs of new, demanding applications such as autonomous vehicles on the road and autonomous transport in the maritime and rail sectors.

    With this contract, Thales Alenia Space will assess the extension of the integrity and safety-of-life services for aviation into the road, rail and maritime sectors. In 2020, the company won the EPICURE project, based on an integrity concept for road travel (tolls and insurance), as well as the IMPRESS project, targeting an integrity service for rail signaling and train separation.

    Thales Alenia Space has been a prime contractor for EGNOS (European Geostationary Navigation Overlay Service) for 25 years. It is a lead industrial contributor to the Galileo system and its ground mission segment and responsible for providing six Galileo Second Generation satellites. In April, the company was awarded a contract to support the implementation and experimentation of the navigation algorithms that will be used in the Galileo Second Generation program.

  • Why radar is the future of autonomous transportation

    Why radar is the future of autonomous transportation

    By Steven Hong, Founder and CEO, Oculii

    Steven Hong, Founder and CEO, Oculii
    Steven Hong, Founder and CEO, Oculii

    Radar has been around since the late 19th century, but today it is poised to revolutionize how autonomous vehicles (AVs) navigate the road. From its nautical origins as a tool to detect the location of ships in heavy fog to being a cost-effective way to prevent collisions in self-driving cars, radar has a wide range of applications.

    For more than 30 years, carmakers and drivers have embedded radar in vehicles to assist with automated cruise control, automatic emergency braking, parking, and more. This effective, hardy technology plays a critical role in the driver experience today, and the same hardware will be used to help AVs navigate the road soon.

    I believe that the next chapter of radar use in vehicles will be in the AV market, where software powered by artificial intelligence (AI) will use radar sensors to read a vehicle’s surroundings and get riders safely to their destination.

    Radar Is a Market-Proven Hardware Solution

    Radar has been around for so long, and the sensors we rely on in our vehicles every day are so reliable, that most drivers are not even aware that they have radar to thank for the assist on their perfect parallel parking job.

    In this era of auto innovation and smart tech, the benefits of turning to this proven hardware solution abound:

    • Radar can perform well in poor weather conditions.
    • It is cost-effective, especially when compared to lidar and camera-based options.
    • Thanks to its low power requirements, adding radar sensors does not significantly impact a vehicle’s energy budget.
    • It is market-proven hardware that is robust and reliable in the field.

    While competing technologies such as lidar are still years away from demonstrating that they can stand up to weather conditions and the toll that mileage takes on equipment, there is no question that radar sensors are up for the challenges of the road.

    The flip side of this coin is that we also have the benefit of knowing the limits of traditional radar technology: It has poor spatial resolution, limited sensitivity, and a narrow field of view. However, this hardware can be greatly enhanced with the right software boost.

    An Oculii sensor placed at the front corner of a vehicle. (Photo: Oculii)
    An Oculii sensor placed at the front corner of a vehicle. (Photo: Oculii)

    Unlocking the Potential of Radar with AI

    Until recently, the best way to improve radar technology was to add more antennas until you got the resolution quality you were seeking. While this approach solves the problem of resolution, it introduces two other problems:

    1. Adding antennas exponentially increases a radar’s complexity, power consumption and size, while only improving performance linearly.
    2. In turn, this added complexity significantly increases the radar’s cost.

    Consider the F-35 fighter jet, which relies on a radar system that costs more than the jet itself. While adding antennas may be a reasonable solution for military-operated airplanes, the consumer AV market would never tolerate the consequent cost increases. However, there is a way that existing automotive radars can be augmented with AI software to improve resolution, without increasing cost, size or power.

    In the same way that AI software transformed what the automotive manufacturers were able to achieve with camera hardware, AI software can revolutionize how radar hardware is used for navigation in AVs.

    Traditional radar sensors emit a constant, repetitive signal that delivers a reliable but low-resolution result. By using innovative AI software to emit an adaptive phase, modulated waveform that changes in real time, the resolution of traditional radar can be increased by up to 100 times. The key to transforming how we use radar hardware is all in the software.

    street view of a driving car (center). At right, the same view is shown with high-resolution radar, with 400+ m of range with precise Doppler/point in all weather conditions. At left, is the view using a standard lidar camera, which has >100 m of range, no Doppler and weather limitations. (Image: Oculii)
    street view of a driving car (center). At right, the same view is shown with high-resolution radar, with 400+ m of range with precise Doppler/point in all weather conditions. At left, is the view using a standard lidar camera, which has >100 m of range, no Doppler and weather limitations. (Image: Oculii)

    Radar with AI

    Reliable sensors with AI software can enable autonomous functions by augmenting the hardware that is already in today’s vehicles. What makes this solution so exciting is that it does not require a design overhaul: the smart sensors in question fit within existing radar packaging.

    Augmenting radar hardware with AI can significantly improve performance while reducing the cost to the consumer. This formula — better performance at a lower price tag — has the potential to greatly accelerate the speed with which AVs make it safely to the consumer market and to revolutionize the automotive industry.

    Rather than pushing forward with the development of costly alternatives that are prohibitively expensive for the consumer market, intelligent radar sensors can bring AVs to the road sooner and for more drivers.


    Steven Hong is the founder and CEO of Oculii, a high-resolution radar company enabling the next generation of autonomous systems. Powered by AI, Oculii software increases the resolution of commodity radar hardware by up to 100 times and works in any environment.

  • How medium-definition maps help navigate dynamic roads

    How medium-definition maps help navigate dynamic roads

    By Ethan Sorrelgreen
    Chief Product Officer, Carmera

    Ethan Sorrelgreen, Carmera
    Ethan Sorrelgreen, Carmera

    Since the early days of autonomous vehicles (AVs), maps — specifically, so-called “high-definition” maps — have played a critical role in their technology stack. Central to perception, localization and path planning, these highly detailed, highly precise maps provide vehicles a baseline understanding of the world around them, delivering key priors that form the basis of the AV’s navigational decision making.

    These maps come with exacting standards: a 3D network graph, spatial accuracy within 10 centimeters, attribute support in the thousands, and so on. Additionally, with AV deployments becoming more frequent — covering broader, more complex driving domains — these requirements are growing ever more demanding.

    Of particular import is the increased need for temporal accuracy — that is, a map’s ability to represent current conditions (as opposed to conditions at some point in time). Roads — especially urban roads — are highly dynamic environments. Things like construction, repaving, signal upgrades and, now, on-street dining constantly affect the flow of traffic.

    For example, in a summer 2020 survey of New York, Carmera found 88 drive-lane-impacting events (out of a total of 251 road events) over 72 hours in midtown Manhattan alone.

    A map’s failure to reflect such events and changes can have a major impact on an AV’s reliability (Will the autonomous-driving feature remain engaged?), motion-planning (Will the AV safely and smoothly navigate through/around the obstacle?) and/or path planning (Will the AV choose the most efficient route despite the obstacle?). Maintaining a map, however, is exponentially more complicated than creating it. Not only does the data need to be good, it also needs to be fast and cheap to produce.

    The key to solving the fast and cheap legs of this classic “good-fast-cheap” trilemma is simplifying the initial problem, using what Carmera calls a medium-definition map. If an HD map is a map with high feature detail and high spatial accuracy, then an MD map is a map with high feature detail but a slightly lower spatial accuracy. It essentially atomizes the dense, complex HD world into discrete, manageable blocks, or “zones.”

    An MD map of a California intersection showing road features — including control attributes — placed with zonal accuracy. (Image: Carmera)
    An MD map of a California intersection showing road features — including control attributes — placed with zonal accuracy. (Image: Carmera)

    These zones — each a logical section of the road network — become the new unit of fidelity. The MD map catalogs all the features in a zone — a traffic light with a left arrow that controls the left two lanes, a bike path, a solid median, etc. — but not their precise location in the real world.

    This simplified map provides the ideal basis for a system of triaging change, which dramatically lowers the cost — in both time and money — of HD map updates. Indeed, it provides the foundation for Carmera’s change-as-a-service offering — a modular, on-demand feed of road events and map updates that plug into existing consumer or HD maps.

    Because of its lower spatial accuracy, an MD map can be updated with consumer-grade tools — a camera and a consumer-grade GNSS, let’s say — coupled with basic consumer vision algorithms. Contrast that to an HD map, which requires either expensive equipment, like a lidar rig, or — in Carmera’s case — sophisticated algorithms that can convert visual and telemetric data into HD road graphs.

    MD map maintenance, therefore, is relatively cheap, which is good news for those who want to use MD data for next-generation consumer applications, such as natural-language navigation, or to support sub-L4 levels of automated driving (both excellent MD use cases).

    An MD map of the same interaction, showing road features—including control attributes—placed with zonal accuracy. (Image: Carmera)
    An MD map of the same interaction, showing road features—including control attributes—placed with zonal accuracy. (Image: Carmera)

    For HD updates, an additional pass is needed. Think of this as a tip-and-cue system: When a functional change in the map is detected (the tip), data from the identified zone is reprocessed using more complex algorithms to create the new HD vectors (the cue). In some cases — either because of customer requirements or because the change is superficial — a simple MD update may be sufficient. Thus, expensive computing resources are only deployed when needed.

    This approach is equally effective for those using traditional lidar-based methods. There, the MD tip allows for targeted dispatching of lidar rigs, which results in significant cost-savings vis-à-vis the typical practice of sequential resurveying.
    As technology evolves, so too will the role of the MD map.

    Carmera sees a world where an AV’s onboard sensors will become so sophisticated that the HD maps’ utility may diminish. MD maps, however, will still provide vehicles key rules-of-the road relationships, helping optimize route planning and similar beyond-line-of-sight decision making. Employing this new standard now, therefore, not only makes driving safer today, it paves the way for the road ahead.

    Screenshot: Carmera
    Screenshot: Carmera