The red ellipses show the MMS orbit paths during the first and second phases of the mission. Each spacecraft uses GPS signals — which come from satellites situated along the green circle shown surrounding Earth — from the far side of Earth to track its position. (Credit: NASA/MMS)
Editor’s Note: See more about the MMS mission in our August issue. The article is also posted here.
News courtesy of NASA
After years of hard work building a spacecraft, a mission team anxiously awaits after a launch — will the instruments they’ve crafted all work as well as planned? This is all the more true when flying new hardware, such as the onboard navigation tool on the four spacecraft that make up the Magnetospheric Multiscale, or MMS, mission, which launched on March 12. This navigation system had never before flown on a spacecraft with an orbit traveling so far from Earth — but if it worked, it would provide the precision navigation needed for MMS.
And the results are now in: Not only has the MMS Navigator system exceeded all of the team’s expectations, it has set the record for the highest GPS use in space.
At the highest point of the MMS orbit, at more than 43,500 mile above the surface of the earth, Navigator set a record for the highest-ever reception of signals and onboard navigation solutions by an operational GPS receiver in space.
At the lowest point of the MMS orbit, Navigator set a record as the fastest operational GPS receiver in space, at velocities over 22,000 miles per hour.
A precise tracking system is crucial for MMS, which requires extremely sensitive position and orbit calculations. The four spacecraft must fly in a tight pyramid formation to gather science data as they move through Earth’s magnetic environment. The formation is required to obtain three-dimensional observations of a phenomenon called magnetic reconnection that occurs when magnetic fields from the sun connect and disconnect with magnetic fields of Earth, which can allow energy and solar material to funnel into near-Earth space. With its instrument booms deployed, each spacecraft is the size of a baseball field — while flying as close as six miles from each other.
Artist’s concept of the MMS (courtesy of NASA).
“Demonstration airplanes like the U.S. Navy’s Blue Angels fly in closer formations, but those planes are also much, much smaller and the pilots are always controlling the movements,” said Brent Robertson, deputy project manager for MMS at NASA Goddard Space Flight Center in Greenbelt, Md. “We have four giant spacecraft each with its own unique orbit that we make maneuvers on about every two weeks. It’s quite challenging to control this formation.”
Tracking spacecraft can be done by radar stations from the ground, but it’s much more expensive and takes longer than an inflight system. However, using GPS as is typically done on Earth by such things as cars, boats and smartphones isn’t nearly as simple for something like MMS. For one thing, the bulk of its highly-elliptical orbit occurs above where the GPS transmitters orbit. So MMS must have specialized, extremely sensitive receivers to capture GPS signals transmitted from the far side of Earth. In addition the MMS spacecraft spin; each one makes three revolutions per minute.
“Spinning adds a whole new dimension to trying to figure out where you are,” said Ken McCaughey, MMS GPS Navigator Product Development Lead at Goddard. “We have four GPS antennas on each spacecraft. As the spacecraft rotates we have an algorithm running that allows us to hand off from one antenna to the next without losing the signal.”
In the first month after launch, the MMS team began turning on and testing each instrument and deploying booms and antennas. During this time, the team compared the Navigator system with ground tracking systems and found it to be even more accurate than expected. At the farthest point in its orbit, some 43,500 miles away from Earth, Navigator can determine the position of each spacecraft with an uncertainty of better than 50 feet.
What’s more, the receivers on MMS have turned out to be strong enough that they consistently track transmissions from eight to 12 GPS satellites — excellent performance when compared to pre-flight predictions that there might be frequent drop outs during each orbit.
Even if the receiver were to lose all GPS signals for part of the orbit, Navigator is specifically designed to handle such dropouts. By gathering as many observations as possible, integrated software called GEONS — for Goddard Enhanced Onboard Navigation System — can still compute the orbit by incorporating additional information including drag force, gravity, and solar radiation pressure.
This system will be even more important during the second phase of the MMS mission when the orbit will double in size and travel all the way out to 95,000 miles from Earth.
“It’s going to be very interesting to see how far out MMS can still receive signals,” said Robertson. “But Navigator has already far exceeded expectations. I think there’s a good chance we’ll end up being able to use GPS and save us some of the expense of using ground observations.”
While Navigator technology and GPS receivers were previously flown for testing and to help navigate a low-earth-orbit mission, this is the first time that the complete Navigator package has been used to actively navigate a high-altitude mission. Now that the team knows it works so well, Navigator can be used for other missions that travel in similar high orbits.
United Launch Alliance crews on July 7 placed the next GPS satellite on top of an Atlas V rocket for blastoff later this month, reports SpaceFlight Now.
Liftoff for the GPS IIF-10 satellite will take place July 15, between 11:36 and 11:55 a.m. EDT, from Space Launch Complex (SLC)-41 at Cape Canaveral, Fla.
This is the second of three GPS constellation replenishment flights planned in 2015. The first launch, of GPS IIF-9 using a Delta IV rocket, occurred on March 25. A third launch, of GPS IIF-11 using an Atlas rocket, is scheduled for Sept. 22. The launch of GPS IIF-12 is scheduled for Jan. 26, 2016.
The Boeing-built GPS IIF-10 is one of the next-generation GPS satellites, incorporating various improvements to provide greater accuracy, increased signals, and enhanced performance for users.
GPS IIF-10 marks the 55th Atlas V launch since the vehicle’s inaugural launch in 2002 and the 27th flight of the 401 configuration. Every operational GPS mission has launched on a United Launch Alliance or heritage rocket. While the government has certified ULA competitor SpaceX for GPS III launches, a SpaceX Falcon 9 rocket exploded June 29, two minutes after launch of an International Space Station resupply mission.
Microsoft will no longer collect its own map data, according to the website re/code. As part of the change, Microsoft is selling some of its assets to rideshare company Uber, including a data center, cameras, intellectual property and roughly 100 engineers. Uber is also buying a data center near Boulder, Colo.
Microsoft plans to continue to offer Bing Maps using data licensed from partners.
Microsoft already gets much of its map data from Nokia and other partners, but had been collecting its own aerial, 3D and street-level maps. It will now source those images from partners, focusing its Bing Maps work on the user experience that overlays the map data and imagery.
Industry watchers suggest the cameras might soon end up on the roofs of Uber vehicles. Uber already has hundreds of thousands of cars being tracked around the world every day.
The ENVI Analytics Symposium (EAS), scheduled for Aug. 25-26 in Boulder, Colo., will bring together experts in remote sensing science to discuss technology trends and the next generation of solutions for advanced analytics.
Geospatial analytics are important because they can be applied to a diverse range of needs in environmental and natural resource monitoring, global food production, security, urbanization, and other fields of research, according to event organizer, Exelis.
The need to identify technology trends and advanced analytic solutions is being driven by the staggering growth in high-spatial and spectral resolution earth imagery, radar, LiDAR, and full-motion video data. Join your fellow thought leaders and practitioners from industry, academia, government, and non-profit organizations in Boulder, Colo., for an intensive exploration of the latest advancements of analytics in remote sensing.
Sponsors and attendees represent organizations such as Airbus Defence and Space, U.S. Fish & Wildlife Services, DigitalGlobe, U.S. Geological Survey, Esri, Naval Post Graduate School, Oak Ridge National Lab and more.
EGNOS Service Provision Workshop 2015 will be held in Copenhagen September 29-30. The workshop is sponsored by the European Satellite Services Provider (ESSP).
The agenda, now available online, includes program and status updates on EGNOS on Day 1, as well as a focus on aviation. Included are an update on the EGNOS Safety-of-Life Service for aviation and several sessions focused on successful EGNOS implementation stories in aviation.
On Day 2, sessions include EGNOS market status and the adoption plan, EDAS for added value applications, E-GNSS benefits in the environmental domain, EGNOS in the maritime application domain and EGNOS in land application domain.
To learn more or to register, go to the ESSP website.
TeleCommunication Systems (TCS) has purchased location-based technology and intellectual property from Loctronix. TCS is integrating the newly acquired assets from Loctronix with its location solution portfolio. Combined, the location-based services (LBS) solutions will enable TCS to further develop indoor-location technology applications ranging from advertising and marketing to navigation and public safety, TCS said.
TCS specializes in secure and reliable wireless communications. The company’s patented solutions enable 9-1-1, commercial location-based services and deployable wireless infrastructure; cybersecurity; defense and aerospace components; and applications for mobile location-based services and messaging.
“Purchasing LBS technology and intellectual property from Loctronix not only adds to our already vast set of intellectual property rights for LBS, more importantly it underpins our continued commitment to further building out our advanced location product portfolio,” said Jay Whitehurst, TCS senior vice president and Commercial Software Group president. “In addition to expanding the breadth of our current location product offering, the Loctronix assets will provide clear differentiators in device location, and increase our market and revenue opportunities — both for commercial and public safety applications.”
“We have exciting news,” Loctronix announced on its website. “The global leader in Precise Device location, TeleCommunication Systems, Inc. (TCS) has integrated our assets with theirs. We have always said that our mission was to locate any device anywhere — indoors or outdoors. With our technology, TCS will now be able to do just that.”
TCS plans to roll out new solutions leveraging the Loctronix assets by year’s end.
Public Works and Government Services Canada (PWGSC) has selected NovAtel’s GAJT-AE antenna electronics for testing on Canadian Army platforms. The GAJT-AE, developed in Calgary at NovAtel’s global headquarters, is a GPS anti-jam solution suitable for small and weight constrained applications. The testing is being conducted through PWGSC’s Build in Canada Innovation Program (BCIP).
PWGSC will procure a number of GAJT-AE’s on behalf of the Department of National Defence (DND). The Director Land Requirements (DLR), with the assistance of the Quality Engineering Test Establishment (QETE), will oversee all testing on DND’s behalf. Field testing is expected to take place in the latter half of 2015 at 4th Canadian Division Support Garrison Petawawa.
The testing will analyze the performance of GAJT-AE on Canadian Army equipment in operational conditions to confirm the suitability and robustness of the NovAtel technology for this role. The process is expected to be completed by the end of March 2016.
GAJT is a null-forming technology that negates jammers, ensuring the satellite signals necessary to compute position and time are always available. Three categories of GAJT are manufactured by NovAtel:
GAJT-710ML: for use with military land vehicles, networks and timing infrastructure
GAJT-710MS: for marine vessels, from small boats to capital ships
GAJT-AE: for use with an external antenna in size and weight constrained applications
“NovAtel has had great success working closely with the Canadian Army on the previous round of BCIP,” said Jason Hamilton, NovAtel’s vice president of marketing. “It is essential to have military users test our products in operational scenarios. We look forward to the valuable feedback that the Canadian Army testing of GAJT-AE GPS anti-jam antenna electronics will provide. NovAtel will use this feedback to continue developing products in support of Canada and its Allied partners.”
The BCIP was created by the Government of Canada to strengthen Canadian innovation. The program offers procurement and testing of pre-commercialized products and services, at a late stage of development. The BCIP:
Bridges the “pre-commercialization gap”
Supports Canadian suppliers by connecting innovators and government users and by testing innovations
Provides real-world evaluation of pre-commercial goods and services
Improves the efficiency and effectiveness of government operations.
Trimble has announced the addition of a Supervisor app to its FieldMaster suite of mobile applications. The latest addition to the suite is designed to enable managers to stay efficient and effective on the go by empowering them to do more work in the field and manage their day-to-day operations remotely. With the FieldMaster Supervisor app, managers can leave the office and still have visibility into their fleet and mobile workers from their smartphone or tablet.
“Mobility is an increasingly valuable tool for field service organizations,” said John Cameron, general manager of Trimble’s Field Service Management (FSM) Division. “It allows a traditional office role to move into the field and still access up-to-date information about the performance of the field operation. The result is a more effective operation where managers in the field have the information they need at their fingertips.”
“I’m often meeting with customers or surveying a job so it’s important for me to know my technicians’ location,” said Dave Rowan, manager of Del-Mar Doors in Delaware. “I use FieldMaster Supervisor whenever I’m on the road. I can open the app and see where my team is, where they’ve been and how long they’ve been at each location. Having quick access to this information has improved my efficiency and productivity. FieldMaster Supervisor is an excellent tool.”
FieldMaster Supervisor is available with Trimble Fleet Management and Trimble Work Management solutions. Features include:
Viewing all locations of your entire team on a map
Seeing each team member’s job progress, including tasks at riskF
Finding the nearest worker to another team member or customer
Navigating to key locations using turn-by-turn directions
Inspecting job performance and documenting status in the field
Receiving important vehicle and driver performance alerts in real-time
FieldMaster mobile applications are a core component of FSM’s integrated suite of field service management solutions that includes Fleet Management, Work Management and Driver Safety. FieldMaster also includes a Technician app, which is an advanced collaboration tool for mobile technicians that allow them to receive, access and update job information in real-time for improved effectiveness on the job.
Figure 1. LightSquared propagation models can underestimate interference by more than a factor of 100 (>20 dB). Two-ray models accounting for ground reflections show LightSquared signals at levels 6 dB above free space predictions several miles out. (Figure from August 2011 GPS World.)
The GPS Innovation Alliance has responded to recent media reports on LightSquared’s position regarding the testing of the compatibility of terrestrial broadband and GPS.
LightSquared is asking the GPS industry to participate in new tests on interference between LightSquared’s spectrum and GPS receivers or waive its right to complain, according to a lawyer representing LightSquared.
Following is the GPS Innovation Alliance’s response:
“The GPS Innovation Alliance (GPSIA) supports a consensus-driven process, including all government and non-government stakeholders, to clearly identify and address remaining technical issues raised by LightSquared proposals to repurpose mobile satellite spectrum for terrestrial broadband use.
“The technical challenges posed by these proposals are formidable, as evidenced by the conclusions of multiple U.S. government entities. Specifically, the U.S. Departments of Defense and Transportation and the NTIA have all found in the last several years that LightSquared’s proposals have significant potential to interfere with GPS.
“Contrary to LightSquared’s recent suggestions, this is not simply a private matter between three GPS companies and LightSquared, but is important to all GPS users who rely on this critical technology every day. The Department of Transportation has sponsored an ongoing effort to assess adjacent band issues, and the GPS industry is actively engaged with the FCC, DOT and other government stakeholders to drive consensus around next steps.
“While we welcome the participation of LightSquared consultants, any further analysis of the technical issues should be informed by input from all of the relevant stakeholders, rather than the one-off efforts of an interested party.”
In a filing last week with the FCC, LightSquared listed 28 different GPS receivers and related devices that it wants to test for interference with its terrestrial mobile broadband service. The devices include certified and non-certified aviation receivers and avionics equipment, general location, cell phones and 13 different high-precision clocks and receivers.
Report from the 2015 China Satellite Navigation Conference
By Greg Turetzky
This May, the sixth China Satellite Navigation Conference (CSNC) was held in Xian, site of China’s famous buried warrior tombs. This was the fourth time I have attended, and every year the event has grown in both attendance numbers and global importance.
The conference opened with the usual provider updates on satellite systems and international collaboration. There was nothing truly unexpected. All the providers continue to make progress towards launching new satellites with new capabilities, as well as providing regional augmentation systems for aircraft navigation.
The hosts were their usual gracious selves and put on a very entertaining evening at “The Night of Beidou” event with wonderful food as well as music, dancing and and acrobats.
Exhibit Hall
The show floor continues to grow at a rapid rate. The program listed122 exhibitors. The market has clearly entered the rapid proliferation stage. The booths were large, well-staffed and busy even during times when technical sessions were in progress. It was hard for me to determine what kind of business was being conducted as there were not many booth staff that spoke English. However, that seemed very appropriate as it was clear that the Chinese domestic market for BeiDou, or BDS, is well established and growing.
In fact, many of the booths were regionally sponsored as there seems to be plenty of local subsidization to grow the GNSS industry in all areas of China. Many companies were displaying end-user products for all segments, from watches to phones to automotive to survey. I also noted significant growth in the number of chipset suppliers; I stopped counting at 10. Of further note and interest, the first few mergers/partnerships have taken place, as the market starts to make its natural turn from proliferation to consolidation.
Technical Sessions
The technical content of the conference is impressive. Approximately 280 papers were presented in up to nine simultaneous tracks over three days. Another 100+ posters were available for viewing.
Here are titles of a few of the papers I liked:
Analysis of relative positioning performance of BDS triple frequency
Anti-spoofing design for Civil Navigation Signal system
Clock-error resolution strategy and precision analysis of GNSS real-time precise satellites
Research on detection and identification methods of satellite navigation RAIM multi-satellite failures.
Research on Wi-Fi/INS indoor pedestrian navigation system based on environmental feature augmentation
Reflections on demands of BDS intellectual property rights in satellite navigation industries
Review of anti-interference RF of satellite navigation receivers
A new TOA estimation method for the navigation pulse of X-ray Pulsare.
If you plan to visit next year, you should consider bringing a translator. Many of the sessions have simultaneous translation, and most of the presentations have both English and Chinese slides, but not all of them. In the past, I have always enjoyed the policy and IP session, but this year it did not have a translator and the presenters spoke in Chinese, so I cannot give you much information. I did notice that several other U.S. companies had sent representatives who were native Chinese speakers.
Conclusion
The Chinese market is now full of grown tigers. I think they worry more about domestic competition for large domestic opportunities than they do about foreigners taking market share from them. That kind of competition has spurred them to catch up quickly in terms of technology and performance to where the big foreign competitors are. I foresee intense domestic competition in the short term leading to fewer, bigger, stronger players who will then be well positioned to compete in the global marketplace.
GREG TURETZKY is a principal engineer at Intel responsible for strategic business development in Intel’s Wireless Communication Group focusing on location. He has more than 25 years of experience in the GNSS industry at JHU-APL, Stanford Telecom, Trimble, SiRF and CSR. He is a member of GPS World’s Editorial Advisory Board. See his previous reporting on the 2014 CSNC, “Tigers, Tycoons on View at China Satnav,” and the 2013 conference, “Little Tigers versus Wolves.”
The statements, views, and opinions presented in this article are those of the author and are not endorsed by, nor do they necessarily reflect, the opinions of the authors present and/or former employers or any other organization the author may be associated with.
A Vision-Aided Integrity Monitor for Precision Relative Navigation Systems
By Sean M. Calhoun, John Raquet and Gilbert L. Peterson
INNOVATION INSIGHTS by Richard Langley
TO MEET THE ACCURACY, availability, continuity and integrity requirements for many navigation applications, multiple-sensor systems are commonly used. For example, a GPS receiver might be combined with an inertial measurement unit, electronic compass and an altimeter to permit enhanced navigation accuracy, availability and continuity in obstructed or otherwise difficult environments. The use of arrays of sensors can also help to ensure that systems used in safety-critical navigation applications provide safe information by maintaining a high level of integrity.
An important group of devices that can be used in multi-sensor systems is one whose processes are based on light. These optical or vision-based devices include laser rangefinders and digital cameras. We could even consider our eyes to be in this group. In common with many other animals, we have built-in visual sensors to get around in our daily lives. Together with our memories, we use our eyes to get safely from one place to another. Ancient mariners tended to sail close to shore so that they could use visual cues for navigation. Later on, they learned how to use the light from celestial objects to navigate in the open ocean. And these days, while we could use the so-called “Mark 1 Eyeball” to continuously monitor the performance of a navigation system, this is often impractical, impossible or unwise.
In this month’s column, we’ll take a look at the development of a generalized vision-aided integrity monitor for precision relative navigation applications. The work is based on the concept of using a single-camera vision system, such as a visible-light or infrared electro-optical sensor, to monitor the occurrence of unacceptably large and potentially unsafe relative navigation errors. A vision-aided integrity monitor of this type could be extremely valuable in augmenting existing precision relative navigation systems, such as GPS, for many different safety-critical aerospace applications such as formation flying, aerial refueling, rendezvous/docking systems, and even precision landing.
It is particularly appropriate that such vision-aided systems be discussed at the present time since 2015 is the International Year of Light and Light-based Technologies, or IYL 2015. This United Nations initiative aims to raise awareness of the achievements of light science and its applications, and its importance to humankind. As mentioned on the IYL 2015 website, “[l]ight plays a vital role in our daily lives and is an imperative cross-cutting discipline of science in the 21st century. It has revolutionized medicine, opened up international communication via the Internet, and continues to be central to linking cultural, economic and political aspects of the global society.”
2015 is also an important anniversary year for several notable developments in our understanding of light. It is the 1,000th anniversary of the work of the Arabic scholar Ibn Al-Haytham, which culminated in his Book of Optics. A Latin translation significantly influenced a number of scholars in medieval and renaissance Europe including Leonardo da Vinci, Galileo Galilei, and Johannes Kepler. 2015 is also the 200th anniversary of Augustin-Jean Fresnel’s proposal that light behaves as a wave and the 150th anniversary of the publication of James Clerk Maxwell’s paper describing electromagnetic wave propagation as we discussed in “Insights” this past March. And we should also mention that 2015 is the 100th anniversary of the publication of Albert Einstein’s general theory of relativity, which includes a description of the propagation of light and other electromagnetic waves in the presence of a gravitational field. And where would GPS and the other global navigation satellite systems and their augmentations be without the understanding that general relativity provides? Nowhere.
“Innovation” is a regular feature that discusses advances in GPS technology and its applications as well as the fundamentals of GPS positioning. The column is coordinated by Richard Langley of the Department of Geodesy and Geomatics Engineering, University of New Brunswick. He welcomes comments and topic ideas. Email him at lang @ unb.ca.
Recently, there has been an increased recognition of GNSS limitations in terms of robustness, availability and interference. As a result of this recognition, there has been renewed interest in developing non-GNSS-based navigation systems to augment system capability. This has become particularly important with the trend toward autonomous systems, where required navigation performance (RNP) metrics, such as accuracy, integrity, continuity and availability become operational drivers. Because of this trend, there is renewed interest in gaining navigational diversity using imaging or vision-aided navigation approaches. Early research with vision systems used 3-D terrain databases and imaging systems to provide periodic position updates in collaboration with onboard inertial navigation systems (INS), much like radar systems did prior to the wide proliferation of GNSS.
For precision relative navigation applications such as formation flying, aerial refueling, rendezvous and docking systems and even precision landing, there is a significant body of research for the use of vision navigation systems. For example, a vision-based relative navigation solution for aerial refueling with the use of an a priori 3-D tanker model has been developed. Results from flight tests showed that image-rendering relative navigation is a viable precision navigation technique for close formation flight, specifically aerial refueling, and demonstrated 95% relative navigation accuracies on the order of 35 centimeters within the operational envelope.
As the body of vision-aided navigation research continues to grow, consideration of other RNP metrics is required. Ensuring that systems are providing safe information and maintaining a high level of integrity is paramount when considering safety-critical navigation applications, but is largely neglected in current vision-navigation research.
The concept of integrity, particularly for navigation systems, refers to the level of trust that can be placed in a navigation system in terms of detecting gross errors and divergences. Many navigation applications have adopted the use of protection levels, which are real-time navigation system outputs that bound the navigation errors to the required probability of integrity risk. For the case of vertical navigation, the vertical navigation system error (NSE) is bounded by the real-time vertical protection level (VPL), and as the long as the VPL is below the vertical alert limit (VAL), the system can continue its operation. Loss of integrity is defined by the case when the NSE > VAL without an alert or, in other words, when NSE > VAL and VPL ≤ VAL.
One of the richest sources of information for how integrity can be handled for precision relative navigation systems can be found with the Local Area Augmentation System (LAAS), which focused on providing integrity under fault-free and single ground reference receiver failure conditions. LAAS employs several quality monitors such as receiver autonomous integrity monitoring (RAIM).
Much of the vision-aided navigation research to date has focused more on system and algorithmic robustness, rather than quantitative and verifiable integrity, particularly for feature-based processing. One approach has introduced the concept of regional bounding for feature correspondence between time-sequenced image frames, including some feature-unique criteria that can provide some protection from feature correspondence errors. Although this approach does yield some robustness for the algorithms, no quantitative integrity characterization was developed. Another approach introduced a truly quantitative integrity monitor for failures in the mapping of features to pixels, particularly in the presence of a bias. This approach predicts the largest possible position error in the presence of one such bias due to feature mismatch using a GPS RAIM-type approach. The current state of research addressing integrity for vision navigation, using an image-rendering or template-matching approach, is even less mature. In fact, we have not identified any previous integrity-specific work for image-rendering vision navigation.
The research presented in this article generalizes the concept of integrity in terms of operating and alerting regions. Applications that use navigation systems generally have objective operating regions that require a certain navigation performance, whether this be around a glide-slope, a formation flight position or even a flight-path clearance. Navigation integrity becomes critical because large divergences from these operating regions, without an alert, can become safety risks. The alert limit is simply the instantiation of this concept. It is the threshold or measure of how much undetected divergence from the operating region can be tolerated without inducing unacceptably large safety risks.
The remaining sections of this article will describe the development of a rigorous and quantitative vision-aided integrity monitor for precision relative navigation systems. First, an introduction to relative navigation using image rendering will be covered in order to describe the fundamental vision navigation approach. This will be followed by a detailed derivation of the proposed vision-aided integrity monitor and simulation based performance results.
Using Image Rendering
The basis of our research is that vision-aided techniques, specifically image rendering, can be used to construct a high-performance integrity monitor for precision relative navigation systems. Image rendering approaches and/or template matching have been used extensively in vision applications such as machine vision, medical image registration, object detection and pose estimation, and recently as a precision navigation system for applications such as aerial refueling and formation flight. The general concept of image-rendering precision relative navigation was evaluated for an automated aerial refueling application, using the approach illustrated in Figure 1. The image rendering approach is based on comparing image sensors with rendered imagery from high-fidelity models, to estimate a relative location based on the best image correspondence.
The image correspondence process is the most critical aspect of the image-rendering or template-matching navigation approach, but the focus of our research is not to make claims of optimality or performance-difference judgments between these image correspondence techniques, but rather show feasibility in the overall vision-aided integrity approach using some of these techniques. Most image correspondence approaches transform the images into feature space, such as scale-invariant feature transform, silhouette, edges and corners, to name a few, and then compute a distance metric between the feature sets, such as Minkowski or Mahalanobis distance, to determine the degree of matching.
Once the actual sensor image is converted to feature space, rendered images are generated based on the relative navigation state estimate using the model, converted to feature space, and compared to the sensor features. This process is repeated across the navigation state space, computing an image correspondence value for each state estimate. The selected navigation state estimate is based on the “best” image correspondence value across the state space.
An example result of this process is presented in FIGURE 2, which shows correspondence values for an edge-based image-correspondence process. In this case, the minimum correspondence value represents the best estimate of the relative navigation state. These image correspondence values between the sensor image (IS) and the rendered reference images (IR) will form the basis for the integrity monitor detection rule.
FIGURE 2. GRD-based image correspondence illustration as a function of 2-D relative navigation state.
Vision-Aided Integrity Monitor Development
As indicated in the preceding sections, our research is based on defining a vision-aided integrity monitor in terms of detecting when the system navigation state (x) is within a specified operating region (XOR) versus being within the alert region state space (XAR). The integrity monitor can yield four distinct conditions: rejection (PR), misdetection (PMD), detection (PD) and false-alarm (PFA). The performance of this type of binary (H0/H1) detection scheme can be characterized using just two of these metrics, the detection and false-alarm rates, which will be the two primary performance metrics for this research. PD is the primary metric measuring navigation integrity, describing the probability that the monitor successfully detects the condition when x ∈ XAR.
Bayesian, Minimax and Neyman-Pearson are a few of the detection schemes available to solve this type of binary detection problem. These detection schemes rely on the knowledge of the underlying statistics of the H0 and H1 condition, often characterized in terms of the probability density functions (PDFs). The main difference between these approaches is the resulting detection rule value (δ). Once δ has been established, the resulting theoretical performances of the detectors are computed by integrating the underlying PDFs of the H0 and H1 conditions, pH0 and pH1 respectively. The probability of detection (PD) is computed as
(1)
The integrity performance of the monitor can also be described in terms of integrity risk or probability of missed detection
(PMD), which is computed as
(2)
Similarly, the probability of false-alarm (PFA) is computed as
(3)
This is represented graphically in FIGURE 3.
FIGURE 3. Graphical illustration of detection performance.
The PDFs represent the statistical distributions of image correspondence values for the respective H0/H1 condition. The general detection rule premise is such that for a given sensor image, the underlying PDF for the “best” image correspondence with the rendered reference set is sufficiently distinct when the sensor image is in an H0 condition versus H1. The characteristics of the H0/H1 PDFs that dictate the monitor performance are dependent on many factors, including the fidelity and accuracy of the world model, the general observability of the image rendering process and the image correspondence approach for the specific application. For our research, we used two image correspondence techniques to evaluate the overall integrity monitor approach.
The first image correspondence technique evaluated is a simple binary silhouette (SIL). In this approach, both the sensor image IS(x) and reference image set IR() are converted to a silhouette using pre-defined thresholds to first convert the red-green-blue (RGB) images to gray scale and then subsequently to a binary image. An image correspondence function computes the percentage of overlap between the silhouettes.
The resulting image correspondence is based on the ratio of the cardinality of these sets. The navigation state estimate () that yields the maximum image correspondence value from the set of rendered reference images or template database is considered the most likely for that particular image sensor (IS).
The second image correspondence utilizes edge features for the image correspondence process. Under this approach, magnitude of gradient (GRD) processing is used, in which the sensor image and the rendered reference images are preprocessed through a Prewitt filter to determine changes in image intensities between adjacent pixels. This process computes the components of the gradient. The gradient magnitude is computed by root-sum-squaring the x-y components and normalized, resulting in an edge detection. A Gaussian blur filter is then applied to the output of the edge detection.
The application of the Gaussian blurring compensates for the spatial discrepancies between the discrete reference set or template database and the sensor image. Finally, the resulting feature images, including both the reference image (IR_GRD) and the sensor image (IS_GRD), are processed through a sum-squared-difference (SSD) image correspondence.
The resulting PDFs are based on the best image correspondence with the RE reference set, which is the minimum for the GRD processing.
These image correspondences build the basis of the detection metric, utilizing both the sensor image (IS) and the rendered reference set (IR), which is spatially distributed across the operating region, illustrated by FIGURE 4. This illustrated example shows instances of both a H0 and H1 sensor image (blue and red, respectively). The underlying H0/H1 PDFs for establishing the detection threshold are determined by sampling sensor images from XOR and XAR and computing the image correspondence against IR. This can be done through a combination of high-fidelity simulation and/or test data. The overall performance of the integrity monitor will be dictated by these underlying distributions. The following sections show the results of this integrity monitor approach for an aerial refueling application.
FIGURE 4. Simplified example of rendered reference set (IR) illustrating image correspondence process for integrity monitoring.
Simulation Evaluation
To explore the performance of the proposed integrity monitor approach, an aerial refueling (AR) application was modeled within a simulation environment. The AR operation lends itself well to the construct of the proposed integrity monitor and is developed to show that the system (refueling aircraft) is in the refueling envelope (RE) and has not violated the alert limit, which in the AR case is the safety boundary (SB). In this operational case, H0 is defined as the condition when the integrity monitor determines the refueling aircraft is in the RE, and H1 as the case when the integrity monitor determines the refueling aircraft to be within the SB. A validity region is also defined in order to bound the problem, in which it is assumed that the refueling aircraft is always within, under both H0 and H1 conditions, as shown in FIGURE 5.
FIGURE 5. Integrity regions of interest for an aerial refueling application and illustrated example of a rendered H0 image set for the refueling envelope used as the correspondence basis for the integrity detection metric.
To determine the underlying H0/H1 distributions, a set of reference images uniformly sampled from the RE was rendered using the associated tanker and camera models. This rendered image set was used as the common basis for performing the image correspondence with the actual sensor image.
The baseline RE reference set used for this research was developed using 504 rendered images distributed in a spherically uniform manner across the entire RE volume. Then, two random sets of simulated sensor images were generated and drawn from both RE and SB regions. It is assumed that the refueling aircraft and corresponding sensor images are within the validity region in order to bound the simulation. This bounding assumption is an acceptable constraint, given that the system most likely had to pass several operational checks to ensure the refueling aircraft is in the general region of the RE as defined by the validity region. To get detailed statistical representation of the PDFs, particularly at the tails of the distribution, both RE and SB image sets included more than 100,000 simulated sensor images, representing true states of the refueling aircraft. The simulation environment for this analysis uses the same refueling tanker model for the sensor images and the RE reference set, which eliminates the effects of modeling errors. Additionally, variations in the attitude are currently not considered. The resulting PDFs for H0 (blue) and H1 (red) conditions are shown in FIGURE 6.
FIGURE 6. Underlying image correspondence distribution for H0 (blue) and H1 (red) conditions.
Figure 6 shows generally good distinction between the H0 and H1 hypotheses — a necessary condition to achieve good detection performance. Several techniques were evaluated for determining the PDF including histogram, nearest neighbor and kernel with a Gaussian weighting function. These underlying H0 and H1 distributions will be used as the basis for designing the detection thresholds, based on the image correspondence of the sensor image with the RE reference set. These results assume uniform prior distributions across the RE and SB regions; however, it would be relatively straightforward to incorporate non-uniform prior information, based on a particular application, as available.
Detection schemes are often characterized using receiver operating characteristics or ROC curves, which illustrate the detection-monitor trade-off between probability of detection and probability of false alarm. The predicted detection performance for this AR application is a function of these underlying H0/H1 PDFs, and this performance is captured in the ROC curves shown in FIGURE 7. The ROC curves show that 10-3 level integrity-monitor detection performance (PD) is realizable for both SIL and GRD image correspondence approaches, while still maintaining a reasonable probability of false alarm (PFA) of less than 0.05 (5%). The SIL approach demonstrates slightly better performance than GRD under the chosen image resolution and RE reference set density. Normally, theoretical ROC curves would extend through the whole range of values [0,1] for both PD and PFA; however, this assumes unbounded PDFs. Doing so would require an infinite number of simulation cases and is obviously not practical for a simulation evaluation to gain statistics necessary to extend the PDFs near the entire theoretical ranges. Overbounding of the PDF tails could be performed to extrapolate and extend the tails of H0/H1 PDFs to determine the integrity detection performance beyond the current ranges, but this was not performed as part of this research.
FIGURE 7. Predicted integrity detection performance for both SIL and GRD image correspondence techniques.
In most applications, conditions exist that are outside of the nominally defined operational envelope, but yet are not significant enough deviations to be considered safety risks that require alerts and action. Such a case exists for the refueling operation under consideration in this research, where there exists a region outside the RE, but not in the SB, which we will refer to as the operational limit volume (OLV). The current definitions of H0 and H1 for the vision-aided integrity-monitor approaches developed above only consider conditions within the RE or the SB volume, and not within the OLV volume. OLV conditions were omitted since they technically aren’t considered a safety or integrity risk. However, it is possible under certain implementations and operational considerations that integrity monitoring coverage is desired under these OLV conditions.
Using the same analysis process as the original evaluation, an updated simulation was performed, this time considering all points within the validity region, including the OLV points. To construct a detection scheme under this new paradigm, the OLV conditions must be either mapped to the existing H0 or H1 hypotheses, or a new hypothesis must be defined, possibly creating an M-ary hypothesis scenario. The approach taken for this research was to consider OLV conditions as a safety risk, which is a conservative approach, rather than defining any new hypotheses. The resulting image correspondence distributions are shown in FIGURE 8. Subplots (a) and (b) show the difference the OLV points have on the underlying PDF distributions. As expected, when the OLV points are excluded, the PDFs track the original distributions quite well. The impact of including sensor locations from the OLV is clear from these figures, yielding a much bigger overlap between the H0/H1 conditions.
FIGURE 8. Simulation testing results assuming OLV states are a safety risk. The prediction represents expected performance without consideration of the OLV states. (a) SIL image correspondence PDFs,(b) GRD image correspondence PDFs, (c) SIL ROC curve, (d) GRD ROC curve.
Much like the PDFs, the ROC curves align with the previous results quite well when the OLV conditions are omitted, but take a order of magnitude integrity performance hit when OLV is captured under the existing H0/H1 definition and detection thresholds. Even under this conservative assumption, the overall monitor performance still yields a 0.96 (96%) detection rate at a 0.05 (5%) false-alarm rate, as illustrated by the ROC curves shown in subplots (c) and (d) of Figure 8. It is likely that these results could be significantly improved by redefining the terms of the H0 and H1 conditions or defining an H2 condition specifically for the OLV region.
Sensitivity Analysis
In addition to the baseline integrity monitor results, various sensitivity studies were performed to evaluate the integrity monitor performance impacts of environmental and hardware considerations. These sensitivity evaluations focused on common vision-based considerations such as sensor distortions and lighting conditions, and monitor design choices such as pixel resolution and reference image density. The sensitivity aspects that were evaluated under this research included the number of reference images, the effects of image distortion, pixel resolution and lighting conditions.
Reference Set Density. In addition to our standard reference set of 504 RE images, we conducted tests using 288 and 729 images. While a larger number of images improves integrity detection performance, processing speed is decreased. It is possible to trade off processing power for performance as necessary for a particular application and the associated integrity monitor performance requirements.
Image Distortion. We applied radial and tangential distortions to the simulated sensor images (IS) such that they represented a 95% certainty of the residual error to represent an outer envelope case for this type of sensor. The impact on the H0/H1 PDFs is very minimal, and the results demonstrate a potential robustness to this common type of sensor effect.
Pixel Resolution. We evaluated eight different pixel resolutions from 12 × 9 to 1280 × 1024 pixels per image. Our results showed a surprising robustness to pixel resolution, indicating only marginal performance impacts down to extremely limited pixel densities.
Lighting Conditions. To explore the impact of lighting conditions, the simulated sensor images (IS) used as the basis for the sensitivity analysis were regenerated under a secondary lighting condition, intended to emulate a much brighter background environment, and processed against the original RE reference set. The results demonstrate that under these varying lighting conditions, the system again demonstrates a high level of robustness, particularly using the SIL image correspondence approach.
Ratio Test Integrity Test
The initial integrity monitor results discussed thus far only used reference images from the operational region, RE. However, it is also possible to use a reference image set created with rendered images from the alert region, SB, by including an additional image correspondence process between the sensor image and rendered SB reference set. This is done to create a ratio test statistic as the detection metric. We compute the ratio of the highest image correspondence between the RE and SB reference sets. This approach is very analogous to the use of ratio tests for GNSS carrier-phase integer fixing.
The resulting ROC detection performance of the ratio threshold approach showed that, as with the single RE reference set, the SIL image correspondence approach yields the best H1 detection performance, resulting in the best integrity protection.
The GRD ratio detection performance also yields improved performance and is comparable to the SIL image correspondence approach solely with RE reference set.
Conclusions and Future Work
In this article, we have discussed the feasibility of a vision-aided integrity monitor for precision relative navigation systems. The research posed the relative navigation integrity problem within the context of an aerial refueling application. Using image rendering, where an imaging sensor and high-fidelity 3-D model is used, we have shown that 10-3 to 10-5 level of integrity monitoring is attainable for aerial refueling and formation flight applications. Having this level of independent monitoring could provide significant relief to a GPS-based precision relative-navigation system from a system-safety and certification perspective. The research demonstrated the proposed integrity monitor was robust against several degrading imaging effects, including lens distortions, lighting conditions and reductions in pixel resolution. Although more work is required to validate the results of this research, which was based on simulated images, the results show high promise for this type of integrity monitor approach.
Disclaimer
The views expressed in this article are those of the authors and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the U.S. Government.
Acknowledgment
This article is based on the paper “Vision-Aided Integrity Monitor for Precision Relative Navigation Systems” presented at ITM 2015, the 2015 International Technical Meeting of The Institute of Navigation held in Dana Point, Calif., Jan. 26–28, 2015.
SEAN CALHOUN is the managing director at CAL Analytics, Columbus, Ohio, and is pursuing his Ph.D. degree at the Air Force Institute of Technology (AFIT), Wright-Paterson Air Force Base, Ohio.
JOHN RAQUET is the director of the Autonomy and Navigation Technology Center at AFIT, where he is also a professor of electrical engineering.
GILBERT L. PETERSON is a professor of computer science at AFIT and vice chair of the International Federation for Information Processing Working Group 11.9, Digital Forensics.
FURTHER READING
Authors’ Conference Paper
“Vision-Aided Integrity Monitor for Precision Relative Navigation Systems” by S.M. Calhoun, J. Raquet and G. Peterson in Proceedings of ITM 2015, the 2015 International Technical Meeting of The Institute of Navigation, Dana Point, Calif., Jan. 26–28, 2015.
Image-Sensor Navigation
“Flight Test Evaluation of Image Rendering Navigation for Close-Formation Flight” by S.M. Calhoun, J. Raquet and J. Curro in Proceedings of ION GNSS 2012, the 25th International Technical Meeting of the Satellite Division of The Institute of Navigation, Nashville, Tenn., Sept. 17–21, 2012, pp. 826–832.
“Fusing Low-Cost Image and Inertial Sensors for Passive Navigation” by M. Veth and J. Raquet in Navigation: Journal of The Institute of Navigation, Vol. 54, No. 1, Spring 2007, pp. 11–20. doi: 10.1002/j.2161-4296.2007.tb00391.x.
“Automated Rendezvous and Docking Sensor Testing at the Flight Robotics Laboratory” by J.D. Mitchell, S.P. Cryan, D. Strack, L.L. Brewster, M.J. Williamson, R.T. Howard and A.S. Johnston in Proceedings of 2007 IEEE Aerospace Conference, Big Sky, Mont., March 3–10, 2007, doi: 10.1109/AERO.2007.352723.
“Performance of Integrated Electro-Optical Navigation Systems” by T. Hoshizaki, D. Andrisani II, A.W. Braun, A.K. Mulyana and J.S. Bethel in Navigation: Journal of The Institute of Navigation, Vol. 51, No. 2, Summer 2004, pp. 101–121, doi: 10.1002/j.2161-4296.2004.tb00344.x.
Simultaneous Localization and Mapping
“A Review of Recent Developments in Simultaneous Localization and Mapping” by G. Dissanayake, S. Huang, Z. Wang and R. Ranasinghe in Proceedings of 6th IEEE International Conference on Industrial and Information Systems, Kandy, Sri Lanka, Aug. 16–19, 2011, pp. 477–482, doi: 10.1109/ICIINFS.2011.6038117.
Navigation Integrity
“Developing a Framework for Image-based Integrity” by C. Larson, J.F. Raquet and M.J. Veth in Proceedings of ION GNSS 2009, the 22nd International Technical Meeting of the Satellite Division The Institute of Navigation, Savannah, Ga., Sept. 22–25, 2009, pp. 778–789.
“From RAIM to NIOAIM: A New Integrity Approach to Integrated Multi-GNSS Systems” by P.Y. Hwang and R.G. Brown in Inside GNSS, Vol. 3, No. 4, May-June 2008, pp. 24–33.
Minimum Aviation System Performance Standards for Local Area Augmentation System (LAAS), DO-245A, by RTCA SC-159 WG-4, RTCA Inc., Washington, D.C., December 2004.
Camera Calibration
“Flexible Camera Calibration by Viewing a Plane from Unknown Orientations” by Z. Zhang in Proceedings of ICCV’99, the Seventh IEEE International Conference on Computer Vision, Kerkya, Greece, Sept. 20–27, 1999, Vol. 1, pp. 666–673, doi: 10.1109/ICCV.1999.791289.
Digital Image Processing
Digital Image Processing, 4th Ed., by W.K. Pratt, published by John Wiley & Sons, New York, 2007.
Digital Image Processing, 3rd Ed., by R.C. Gonzalez and R.E. Woods, published by Prentice Hall, Upper Saddle River, N.J., 2007.
Signals and Noise
Detection of Signals in Noise, 2nd Ed., by R. N. McDonough and A.D. Whalen, published by Academic Press, Inc., Waltham, Mass., 1995.
An Introduction to Signal Detection and Estimation, 2nd Ed., by H.V. Poor, published by Dowden & Culver, an imprint of Springer, New York. 1994.
