GPS World

Tag: Insurance Institute for Highway Safety

  • Safety testing in indoor and challenged environments

    Safety testing in indoor and challenged environments

    A GPS-like ground-based technology teamed with inertial measurement and driving robots to deliver the necessary accuracy when obstructions knocked out GPS as a reliable sole sensor.

    By David Aylor, Insurance Institute for Highway Safety
    Andrew Pick, Anthony Best Dynamics Ltd.
    Paris Austin and Martin Parry, Oxford Technical Solutions Ltd.

    Consumer information organizations like the Insurance Institute for Highway Safety (IIHS) design test procedures to compare different automobile manufacturers’ safety systems. The test equipment must be repeatable and as independent as possible of time of day, weather conditions or test-driver behavior.

    In 2015 IIHS completed a $30 million expansion of the Vehicle Research Center (VRC), its centerpiece a 5-acre fabric-covered track, to allow testing to continue rain or shine. It is complemented by an outdoor track for a total area of 15 acres.

    IIHS rates crash prevention systems such as Forward Collision Warning (FCW) and Automatic Emergency Braking (AEB), and looks at how well those systems can identify road users like pedestrians and bicyclists.

    To simulate real-life potential crashes for safe, accurate and repeatable testing, the Institute has been researching robotic equipment to automate some of the driving tasks.

    While the covered track offered much needed all-weather testing capability, it introduced a challenge for the standard high-accuracy GPS/GNSS equipment used for testing. IIHS operates a multi-frequency GNSS base station with real-time corrections. High-accuracy position, velocity and time (PVT) and other relevant parameters from these GPS units are required for testing and are essential for operating robotic test equipment.

    However, tests on the covered track clearly showed the equipment was not delivering the required accuracy, reliability and repeatability: the steel trusses of the covered track roof were a sufficient obstruction to GNSS signals.

    Locata. Locata provides an RTK GPS-like positioning capability utilizing ground-based transmitters which precisely time-synchronize to one another using their proprietary ranging signals without the need for cables or atomic clocks. This delivers centimeter-level accuracy with very high reliability, in networks of strategically placed, static LocataLites (LLs).

    The IIHS Locata network was deployed with 16 LLs covering both open and covered test tracks (Figure 1). The network meets two key requirements: accuracy of 10 cm or better at 95% confidence and a very high degree of repeatability with a service availability (defined as meeting the above requirement) of better than 95% of the time.

    FIGURE 1. VRC Locata Network and HDOP Quality in Locata Service Area. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 1. VRC Locata Network and HDOP Quality in Locata Service Area. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)

    AB Dynamics. Anthony Best Dynamics supplies driving robots for the design, development and testing of automotive technology. Driving robots precisely and accurately control the vehicle steering wheel, brake and throttle pedals with a level of repeatability that vastly exceeds that achieved by human test drivers. When coupled with an accurate position measurement sensor the possibility of centimeter accurate path-following control becomes reality.

    In ABD path-following control software, motion data is collected from a Locata/INS integration unit at 100 Hz and fed back to the robot’s path-following controller. The path-following controller employs a speed-dependent look-ahead algorithm that not only maintains the vehicle heading but allows centimeter-accurate path control.

    OxTS. Oxford Technical Solutions specializes in the design and manufacture of GNSS-aided inertial navigation systems (GNSS/INS) for automotive testing.As well as one-centimeter position accuracy, OxTS systems measure movement in all vehicle-axes at up to 250 Hz.

    Systems that only rely on inertial measurements are also prone to drift with time, so OxTS products are GNSS-aided; several other inputs can be used alongside the inertial measurement platform to create a hybrid system where each technology mitigates weaknesses in others.

    The Locata network and associated receivers are configured to use the same time and coordinate frame as GPS so the measurements are identical to that of a GPS receiver. The OxTS system then uses this information as it would normally and is able to output accurate and reliable vehicle measurements while maintaining excellent position accuracy.

    Measurements can be utilized by other equipment such as driving robots or logged for post-processing. Raw measurements are also logged internally so the data can be downloaded and reprocessed post-test, to test different scenarios or make other changes.

    The driving robots have steering and pedal actuators that can be quickly installed without the need to make modifications to the vehicle as shown in Figure 2. Even with the robots installed, the steering wheel, throttle and brakes remain accessible to a human driver. At the heart of the robot is a dedicated real-time controller, which coordinates the steering and pedal robots and captures data at 1000 Hz.

    FIGURE 2. Driving robot. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 2. Driving robot. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)

    Locata and OxTS units were installed in a rear passenger seat. The Locata antenna was roof-rack-mounted on a ground plane, approximately aligned with the centerline of the vehicle. The roof rack contained a second Locata antenna connected to a second Locata receiver. This was used for post-processing accuracy analysis of the fixed baseline (distance) between the two Locata antennas.

    Test procedure

    The automation kit enables the vehicle to be driven in manual mode and record scenarios for later replay. Drive scenarios can also be created in the user interface using basic geometric shapes and designate start, end or special maneuvering points within drives.

    A local two-dimensional coordinate frame can be created with or without alignment to a global coordinate system. Each scenario may be replayed at various speed settings. For instance, most scenarios described later were replayed multiple times at different speed settings, often incrementing in fixed steps from a low speed such as 10 Km/hr.

    The demonstration platform was driven in various driving patterns on both test tracks. Figure 3 shows these patterns as a map derived from reported vehicle positions during the repeats of each scenario.

    FIGURE 3. Test Scenarios.(Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 3. Test Scenarios.(Figure: D. Aylor, A. Pick, P. Austin and M. Parry)

    The Double Lane Changes (DLCs) conducted on both tracks resemble the driving pattern needed for testing most collision-avoidance and lane-change features. The S-curve is a driving pattern used for the IIHS headlight evaluations.

    Analysis and results

    Data analysis was focused on characterizing the accuracy and repeatability of the automated test setup as a complete system first and then Locata alone as the core positioning system. As the first step, data from two full days of testing were reduced to repetitions of the various driving patterns shown in Figure 3. Start and end times of each repetition were extracted from AB Dynamics systems and corresponding Locata system data was further processed to generate the results shown here.

    The foundation for highly repeatable control system and positioning accuracy is to have a highly reliable network that delivers repeatable DOPs and number of ranging signals at any given track location. Repeatability of the numbers of LLs seen and the HDOPs were investigated for this purpose. Shown in Figure 4 is the actual number of LLs observed and the resulting HDOP during the five repeats of the DLCs done at 45 km/h in the covered track.

    FIGURE 4. HDOP & LL Count in Double Lane Change at 45 km/h (Covered Track). (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 4. HDOP & LL Count in Double Lane Change at 45 km/h (Covered Track). (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)

    The number of LLs used remain constant at seven as expected and the HDOP change resulting from the motion repeats for each of the repetitions. Shown in Figure 5 are similar plots for the seven repetitions of the Lap scenario done at 20 km/h in the open track. In these, the LLs used vary between 8 and 9, with the drop happening at one end of the lap. Although slight variations can be seen in the times of the drops due to the varying speed of the vehicle during the turns, the HDOP pattern repeats consistently for all seven repetitions.

    FIGURE 5. HDOP & LL Count in Lap at 20 km/h (Open Track). (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 5. HDOP & LL Count in Lap at 20 km/h (Open Track). (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)

    Analysis of the 48 DLC repetitions from the covered track is presented in Figure 6. Locata position data from all repetitions were averaged along the drive path to estimate a best fit path and the deviation from this was estimated (top subplot). The best fit path allows the estimation of the run-to-run deviation of the vehicle path. The middle subplot shows the mean and standard deviation of cross track error (or spread) of all the repetitions compared to the best fit path.

    FIGURE 6. Covered Track Double Lane Change Performance Statistics. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 6. Covered Track Double Lane Change Performance Statistics. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)

    Despite the 48 DLC repetitions being carried out across a range of speeds (10-45 km/h) a high level of repeatability was measured. In straight segments the control system was able to repeat all the runs with below 4 cm of mean deviation from each other. This increases to 5 cm during turns due to the increasing lateral acceleration at higher speeds. The standard deviation also follows the same pattern, remaining below 3 cm during the straight-line segments and increasing up to 5 cm during the turns. The bottom plot shows the mean and standard deviation of the baseline error measured between the two Locata antennas on the vehicle.

    Locata baseline error from repetitions of all scenarios were then used to estimate a probability distribution function (PDF) to assess the Locata positioning system performance alone. This included close to 180,000 data points from around 5 hours of automated driving in various parts of the IIHS tracks. Resulting PDF is shown in Figure 7.

    FIGURE 7. [Brown] Locata position accuracy ±3 cm (95%) using the fixed baseline between two independently operating antenna-receiver pairs in the vehicle (5 hrs of automated driving on both tracks). [Blue] ABD system repeatability ±6 cm (95%) using across track error from 48 repetitions of the Double Lane Change maneuver on the Covered Track. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 7. [Brown] Locata position accuracy ±3 cm (95%) using the fixed baseline between two independently operating antenna-receiver pairs in the vehicle (5 hrs of automated driving on both tracks). [Blue] ABD system repeatability ±6 cm (95%) using across track error from 48 repetitions of the Double Lane Change maneuver on the Covered Track. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    This baseline error PDF gives a positioning accuracy of ±3 cm at 95% for the Locata position system, exceeding the IIHS requirement for positioning of 10 cm at 95% (Figure 8). The control system repeatability itself shows ±6 cm at 95%, better than IIHS expectation for positioning system alone.

    FIGURE 8. Covered track automated double-lane change (DLC) test. Fully automated path following with two back-to-back lane changes through traffic delineators set 15 cm from the sides of the vehicle. Drop-in control system repeatability of ±6 cm (95%) achieved using Locata positioning accuracy of ±3 cm (95%) through 48 repetitions at speeds ranging from 10 to 45 km/hr. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)
    FIGURE 8. Covered track automated double-lane change (DLC) test. Fully automated path following with two back-to-back lane changes through traffic delineators set 15 cm from the sides of the vehicle. Drop-in control system repeatability of ±6 cm (95%) achieved using Locata positioning accuracy of ±3 cm (95%) through 48 repetitions at speeds ranging from 10 to 45 km/hr. (Figure: D. Aylor, A. Pick, P. Austin and M. Parry)

    Conclusion

    The IIHS, one of two organizations in the United States that issue public crash safety ratings, is using Locata, a GPS-like local positioning system, under a canopy-covered test track that doesn’t have RTK-capable GNSS signal visibility.

    Precise positioning from Locata integrated with INS by OxTS demonstrates automated path following with centimeter-level repeatability using driving robots from AB Dynamics. The authors thank and acknowledge the Locata team for the excellent support provided throughout the project.

  • Robot: Target on Its Back

    Robot: Target on Its Back

    Photos: Brian J. Geiger.
    Photos: Brian J. Geiger.

    Two Autonomous Vehicles Seek Safe Avoidance in Critical Tests

    A new state-of-the-art research center runs car-makers’ safety systems through their paces, in tandem with a soft-target robot that can be crash-impacted without adverse effects. Precise positioning and exact repeatability of test sequences are key criteria.

    Paul Perrone, Perrone Robotics

    The Insurance Institute for Highway Safety has undertaken a $30 million expansion project at its Vehicle Research Center near Washington, D.C., enlarging and enhancing a state-of-the-art vehicle test track and building a new 700 x 300-foot (213 x 91-meter) covered track for weather-resistant testing.

    The VRC will use new robotic and positioning technologies to achieve required levels of precision and repeatability for vehicle testing of frontal collision avoidance and other safety systems. Tests of both the same and different vehicles must be conducted under  identical, controlled conditions for the results to have comparable fidelity.

    Crash tests and research conducted at the VRC help drive life-saving improvements in vehicle designs. The new facility will enable staff to evaluate emerging automated vehicle technology in commercial vehicle systems intended to prevent crashes or lessen their severity, with the goal of encouraging the entire industry to adopt the most effective new features.

    Safety systems in vehicles to be tested include the following:
    ◾    Adaptive cruise control
    ◾    Collision-imminent braking
    ◾    Lane-departure warning/correction
    ◾    Other automated technologies.

    Such functions represent semi-automated functions aboard vehicles now on the road. The system is also designed to address and test the full spectrum of semi- to fully-automated vehicles, addressing evolving levels of autonomy and ultimately producing driverless vehicle technology.

    IIHS has contracted Perrone Robotics, Inc. (PRI), to deliver a robotic system for testing such vehicles. PRI develops new applications using its MAX robotics and suite of automation software building blocks. MAX enables rapid integration of a range of sensor and actuator types and has evolved with several frameworks, including MAX-UGV for unmanned ground vehicles. PRI has used MAX-UGV to build automated passenger cars, all-terrain vehicles, tractors, custom platforms, and rockstar Neil Young’s long-range electric LincVolt, a converted 1959 Lincoln Continental.

    FIGURE 1. PRI test system.
    FIGURE 1. PRI test system.

    For the first phase of the IIHS project, the Perrone Robotics system includes a robot target vehicle with the footprint of a car, but measuring only 4 inches high, with a 1-inch ground clearance.

    A robot target vehicle with the footprint of a car measure 4 inches high.
    A robot target vehicle with the footprint of a car measure 4 inches high. Photos: Brian J. Geiger.

    Test Scenario Example. One instance to be tested is National Highway Traffic Safety Administration criteria for crash-imminent braking (CIB). The CIB concept goes beyond the forward-collision warning systems already found in many new cars by actually engaging the brakes when a driver, at fairly slow speeds, gets too close to the car in front of him. In tests, the while the test vehicle travels at similar speeds on a programmed collision course with the robot.

    The target robot vehicle carries one of a number of soft targets. If the vehicle under test fails to prevent a collision with the robot target, the test vehicle runs over the robot target vehicle, dislodging the soft target, but avoiding damage to the test vehicle, robot target vehicle, and the soft target. The next phase of the project adds smaller-footprint target robot platforms with soft targets, representative of pedestrians and cyclists.

    To ensure that the test vehicle can perform repeatable tests, the system also includes a drop-in actuator kit that can be installed into any test vehicle in 30 minutes or less. The system is designed to allow a human driver to sit comfortably in the vehicle and optionally drive, but can also control the throttle, brake, and steering to drive test profiles. Repeatability is key for the operation of robots and vehicles, as well as track conditions, which will be helped by the covered track.

    The VRC test track is installing Locata as its positioning system. In addition to alleviating concerns about GPS outages or dead/weak signal spots, this enables the PRI system to be operated reliably inside the new covered test track. While GPS is not an option for covered or indoor test sites and suffers from environmental issues, the high fidelity and localized positioning provided by Locata overcomes these barriers to test.

    Drop-in actuator kit steering.
    Drop-in actuator kit steering. Photos: Brian J. Geiger.
    Drop-in actuator kit throttle-brake.
    Drop-in actuator kit throttle-brake. Photos: Brian J. Geiger.

    PRI will deliver the target robot and drop-in actuators custom-built. The company looked at starting with existing platforms and building from them, but it would have been infeasible or overly expensive to meet the IIHS requirements for this system. Most existing systems were developed for vehicle dynamics testing or low-speed/simple collision testing. Most couldn’t handle some or all of the more challenging requirements such as the following:

    Drop-in actuator kit:

    • Allow human driver to sit comfortably and drive the vehicle without interfering;
    • Drive autonomously while also allowing for hybrid modes whereby test drivers and onboard systems may assist or take over controls;
    • Offer out-of-the-box programmability and flexibility to handle a wide range of test scenarios and automated vehicle levels;
    • Install into any vehicle in 30 minutes or less;
    • Do not damage vehicle with installation; retain a significant percent of resale value.

    Target robot vehicle:

    • Accelerate from 0 to 55 mph in 10 seconds;
    • Survive collisions at speeds up to 55 mph;
    • Allow collision-avoidance testing with minimal damage to test vehicles and target robot;
    • Scale for carrying a wide variety of soft-target payloads and enable a wide range of vehicles, from small car to SUV to tractor-trailer) to be tested.

    Locata positioning system:

    • Work outside and also on covered track; cover track area with no dead/weak spots;
    • Deliver better than 10-centimeter accuracy for position measurements and relative position control of robots and vehicles;
    • Deliver position updates at 100 Hz in combination with attitude and heading reference system (AHRS) or inertial navigation system (INS).

    The positioning requirement derives from the testers’ need not only for accurate location data of each vehicle, but for precise knowledge of how far apart they are while performing real-time control to orchestrate repeatable scenarios, intersecting vehicle and robot paths to determine whether the vehicle acts to prevent a collision.

    A human operator is easily accomodated within the drop-in actuator kit.
    A human operator is easily accomodated within the drop-in actuator kit. Photos: Brian J. Geiger.

    Safety Systems

    There is a driver in the test vehicle, and there are personnel present at the test site who could be injured by a test vehicle or target robot platform. In addition to wireless e-stop remotes, the test vehicle and target robot systems can be disabled and stopped by a number of events. In the target robot, an e-stop causes the battery pack to be completely disconnected from all vehicle systems, and a spring-load is released applying mechanical braking to stop the vehicle.

    Under normal conditions, the spring is held back with a pneumatic system and air is dumped upon e-stop event. A target robot e-stop is triggered by
    ◾    remote e-stop controller
    ◾    a command issued by control software
    ◾    loss of communication with external systems
    ◾    failure of or loss of communication with internal systems
    ◾    loss of power.

    Aside from fail-safe remote and onboard e-stop systems, additional safety measures are employed by means of safety controllers that monitor safety-critical regions of software, implement a wide range of robot-safety checking rules, and ensure that the robot is operating within safe parameters of the environment (such as by staying within an invisible fence and pre-defined operation boundary).

    Common MAX-UGV Robot Logic

    A common instrumentation and control system (CICS) for both the target robot platforms and test vehicle instrumentation and robotic assist platforms is illustrated in Figure 2.

    FIGURE 2. Target robot logic flow.
    FIGURE 2. Target robot logic flow.

    Embedded Controller. The heart and soul of the vehicle hosts and runs the algorithms, receives sensor data, and executes actuation commands to the motor controllers based on desired route plans and dynamic sensor information. This controller runs the MAX software platform, MAX-UGV framework, and various MAX drivers.

    I/O Controller. Handles inputs from sensors for temperatures, voltages, and currents as well as monitoring limit switches and actuating relays. Certain controls are planned such as mock brake lights on target robots and warning lights in test vehicles.

    Locata. A constellation of nine LocataLite units on towers covers the existing track for Phase I of the project. Phase 2 will require additional units to add coverage to the covered track; some towers will provide coverage to both tracks. Each target robot vehicle and each drop-in kit for the test vehicles carries a Locata rover.

    Locata’s new software essentially adds some capability from its indoor software to its outdoor software to deal with reflections/multipath issues caused by the metal buildings at the test site. The new software also allows the rover to perform real-time calculations on board, required for the less-than 10-cm accuracy. Previously this had to be done on a separate system and data had to be transferred back and forth, which worked against meeting real-time position update requirements for controlling speed, position, and relative position of robots and test vehicles. In test vehicles and target robots, the Locata rover position updates are merged with the output of an attitude, heading, and reference system (AHRS).

    AHRS. The CICS in the robots and test vehicles includes an AHRS that provides the required heading, position, and velocity updates. Accuracy requirements are heading, 1 degree; position, less than 10 centimeters; velocity, 1 mph. Our required position update rate is 10 Hz. We expect to achieve 100 Hz in our system, which improves self-nav capability and overall performance. This rate also applies to other measured/logged data. A Kalman filter computes data from sources within the AHRS and from external sources: GPS and Locata.

    Wireless Adapter, Antenna. On our critical channel, we exchange messages between vehicles to effect proper trajectory and relative positioning. Our e-stop controllers and safety systems also use this network. The non-critical channel is used for test setup and supervisory control, decimated data transmission for HMI monitoring, and logged data transmission.

    Wireless E-stop Interface. This interface is for remote shutdown of a vehicle. The e-stop triggers are similar for the test vehicle systems, but the driver can also disable the robotic system. Rather than brake the test vehicle, an e-stop of the test vehicle systems disables the steering, brake, and throttle actuators into limp-mode and releases control of the test vehicle to the driver.

    Safety Controller. A separate watchdog controller monitors live conditions and the embedded controller and onboard systems, and serves as a direct line for remote wireless e-stop.

    Electronics, Motors. These includes six high-performance 4-inch motors, motor controllers, cut-off contactors, and overall cut-off system for e-stop.

    Conclusion

    The IIHS expansion project is a first of its kind for automated vehicle testing, breaking new ground for target positioning and control, and providing the first indoor test track for this purpose. Data from these tests will be used to improve safety of on road semi- and fully-automated vehicles and help save many thousands of lives, setting a high bar for capability and performance of all automated vehicle functions. Requirements for safety, repeatability, and seamless handoff between driver and autonomous control of the test vehicles, as well as the speeds at which the robots must travel and survive collisions, pose significant challenges. We believe our systems meet them fully.