In recent years there has been a proliferation of software-defined radio (SDR) data-collection systems and processing platforms designed for GNSS receiver applications or those that support GNSS bands. For post-processing, correctly interpreting the GNSS SDR sampled datasets produced or consumed by these systems has historically been a cumbersome and error-prone process.
This is because these systems necessarily produce datasets of various formats, the subtleties of which are often lost in translation when communicating between the producer and consumer of these datasets.
This specification standardizes the metadata associated with GNSS SDR sampled data files and the layout of the binary sample files.
The GNSS SDR Metadata Standard defines parameters and schema to express the contents of SDR sample data files. The standard is designed to promote the interoperability of GNSS SDR data collection systems and processors.
The metadata files are human readable and in XML format. A compliant open source C++ API for reading metadata and binary samples is also officially supported to promote ease of integration into existing SDR systems.
Review the formal standards document and click on Submit a Comment to provide feedback. Comments will be accepted through December 31, 2017.
By Denis Parrot Guest Columnist, Survey Scene newsletter
I recently read an article by David Doyle entitled “Why Doesn’t my Centimeter Match Your Centimeter?” (May Survey Scene), which painted an interesting portrait of the widespread use of GNSS-based data collection systems. Nowadays, almost everyone can claim to have surveys with centimeter or sub-meter accuracy anywhere in the world, which before was achievable only by a few rare specialists. In many cases, the positions do not always fall at the right place when they are integrated into geographical information systems (GIS). Unfortunately, it is often at this stage that we call on a specialist! The key is in the reference system, also known as datum.
Even if it can become relatively complex when it comes to details, the concept of a reference system when using the GNSS system is essentially identical to conventional surveying techniques. A surveyor always used a reference station for his survey using his theodolite. He stringently maintained his control point network on the territory being surveyed. All his surveys were in reference to his control network (points that are part of a series of controlled polygons). If certain polygons were not well linked to the rest of the network, integration with centimeter precision into the computer-assisted design program (or his geographical information system [GIS]) would be problematic, or even impossible, for the lots linked to the erroneous polygons. And this is exactly what happens with the GNSS system; if the reference is overlooked, GIS integration becomes a real problem.
Without realizing it, GNSS users also use a reference network. If they work in RTK (differential positioning), they use either a VRS (Virtual Reference System) or a station network that sends corrections. In general, these two systems are based on a national reference system (geocentric or not). Users can also use a portable base that they install themselves as needed. When they enter a coordinate into this portable station, they become responsible for the reference system used. During post-processing, the problem remains the same. A position must be entered for the base station and the reference must be known (i.e., the datum).
Example of positions that do not fall at the right place when they are integrated into a geographical information system (GIS).
If they use satellite-based augmentation systems (SBAS), such as the private Omnistar, Veripos or StarFire systems, these systems are generally referenced by a geocentric system (which nowadays includes a temporal drift) defined by international bodies (IGN in France, which maintains, along with several research centers, the International Terrestrial Reference Frame (ITRF)). Today, this method of positioning is known as PPP or precise point positioning.
These different GNSS positioning methods, therefore, do not use the same reference! Each may provide highly accurate coordinates. However, these coordinates are only accurate with regard to their reference. Although this concept may seem very simple, in reality the increasingly common use of these systems by non-specialists often produces strange results.
Below are a few examples of uses that may lead to confusion with reference systems, in terms of “compared with what?”:
A farmer who carries out micro-topography to analyze his fields is pleasantly surprised by the level of altimetric precision he can achieve (within a few cm), using a “single-frequency” receiver. However, when he tries to juxtapose two fields, he may discover an altimetric deviation of up to 2 meters. The reason for this is quite simple: he systematically used local base stations with average coordinates taken in the field as the base coordinates. Normally, he uses one base per field. By not taking into account the consistency of the coordinates for the base stations, without realizing it, this farmer was creating independent references for each of his fields. It is obvious that if the analysis had been done individually on each field, he would not have seen any problem. It is when the two juxtaposed fields are integrated that problems arise.
Photo courtesy of Effigis.
In Quebec, a surveyor uses a Department of Energy and Natural Resources station as a source for RTK differential corrections. This system is referenced to the NAD83 SCRS coordinates system. When he tries to integrate his survey points using the conventional method, which are based on the original NAD83 coordinates system, he will notice inconsistencies of a few to several centimeters. These two coordinate systems (NAD83 ≠ NAD83SCRS) may have inconsistencies of several centimeters from each other.
In Canada, an agronomist surveys the position of trees in the city using a portable GNSS system with an SBAS corrections system. Once the survey has been completed and he integrates the position of the trees into the city’s GIS, these all seem to be off by +/- 1.5 m. The SBAS reference systems are all geocentric (within +/- several cm) compared with the NAD83 SCRS system, which is positioned +/- 1.5 m from the Earth’s center!
These three examples illustrate the type of error commonly found in the “reference system” category. The issue here is not GNSS system performance, since we are assuming that the positions obtained are accurate versus the chosen reference. Neither does the issue concern the variations between different reference systems, or the methods or precision of transformations between one reference system and another (this topic is very broad).
Lastly, we should find the answer to the question “But compared with what?” in what we commonly call “metadata.” Regarding GNSS position, this is information related to creating positions: the reference system used, statistics regarding the position estimate, date, different dilution of precision (DOP) values, type of GNSS signal used, etc.
For your next delivery, simply mention “Precise, but compared with what?” or deliver your positions along with full metadata!
Denis Parrot earned a degree in surveying and mapping from Université Laval in 1981, and began his career working in those fields around the world for five years. He then completed graduate studies in Fredericton, New Brunswick, where he earned a master’s in satellite geodesy. With a passion for his area of expertise, Parrot has been involved since 1991 in a host of projects that employ geospatial information to meet the specific needs of various markets and users. He is currently president at Effigis and responsible for the OnPOZ Product division, including the commercial aspect and R&D activities.
This column originally appeared as a blog at the Effigis Geo Solutions website, and is shared with permission.
Editor’s Note: This month, we introduce a column by David Doyle, one of our two new survey editors. Doyle brings to GPS World more than 40 years of experience as a geodesist and surveyor with the National Geodetic Survey — see his full bio at the end of this article. He will be joined by coeditor Dave Zilkoski, who will contribute the June column.
David Doyle
Since the mid-1980s, thousands of articles have appeared in peer reviewed journals, trade magazines and professional organization publications that describe the phenomenal capabilities of contemporary space-based positioning systems. The majority have been about various uses of the United States Global Positioning System (GPS) and increasingly include the potential for the inclusion of the Russian GLONASS, European Union Galileo and China’s BeiDou collectively referenced as Global Navigation Satellite Systems (GNSS).
Without meaning to understate the process, the ability for almost anyone, anywhere, at any time to determine a three-dimension position accurate to within a few centimeters is well established. I often comment that the systems are generally so easy to use that if you have the IQ of a squirrel you can obtain pretty good quality data. A feat that until recently was achievable only by the small community of geodesists and geodetic surveyors is now a near-trivial process for anybody who can make a modest investment in some form of positioning system device — and it’s getting better, faster, cheaper and more accurate all the time.
Our ability to collect, manage and display monumental amounts of positional data is also enhanced by the advances in Geographic Information Systems (GIS).
I have been privileged to be a part of this revolution since my initiation into the world of geodetic positioning in 1967, courtesy of the Selective Service System and the U.S. Army, and their use of geodetic triangulation combined with emerging artificial satellite systems such as SECOR (Sequential Collation of Range). These introductions to geodesy eventually led me to a position with the National Geodetic Survey and a career that spanned 40+ years.
During that time, we watched as the centuries-old method of triangulation was replaced by GPS, and as the prices of equipment plummeted with the integration of this technology into a multitude of public, private and academic disciplines — everything from geophysical sciences to weather prediction, precision agriculture, improved marine and aeronautical navigation. The list goes on and on and is well known to those who read this magazine.
So where is this going? What does the title of this article mean?
Doyle working on the Washington Monument.
If we accept, which we do, that all these things are true, then why is it that the world of sharing positional information is filled with scenarios that go something like this? “I got a cm and you got a cm, but our centimeters differ by a meter.” What this means is if these systems are so capable, then virtually all positional data integration should be a snap — everything should fit like a bespoke shirt. Unfortunately, that is often just not so.
Take the case of a decree issued by the U.S. Supreme Court in December 2014 delineating the offshore boundary between the United States and the state of California. The boundary is defined as a set of Universal Transverse Mercator (UTM) grid coordinates published to the nearest mm and referenced simultaneously to the North American Datum of 1983 (NAD 83) and the World Geodetic System 1984 (WGS 84), which the decree states are interchangeable. At the mm level, this is not true. In this area, they differ by approximately 1 m.
The decree provided no information on how these positions were derived, how accurate they really are, and who performed the computations — it certainly was not the Supreme Court. Without pointing fingers at the responsible agency, these all-too-common occurrences seems to be rampant among the many users of high-accuracy positional data, both horizontal and vertical. The crime is often the sin of omission.
The failure in many cases is a lack of knowledge on the part of many GNSS users of some of the basic principles of geodesy and geodetic surveying guidelines and providing complete metadata such as:
what geodetic datums and potentially which realization of those datums were referenced?
what are the units of measure?
how accurate are the positions/heights really?
It’s important to note that the number of digits to the right of a decimal point have nothing to do with accuracy. Land surveyors are taught from their first day on the job that they are following in the footsteps of the surveyor that went before them. It is not unusual for surveyors to struggle with incomplete information from previous surveys to be able to make accurate interpretations of what the original or other previous surveyors intended — a lack of complete metadata.
Today, the massive amount of coordinate and height information being generated by thousands of surveyors, engineers and other disciplines are those footsteps — albeit digital. The multitudes of high-quality data being collected around the world is only as good as the associated information about those values.
As we rapidly approach a time when there will be vastly improved GNSS constellations and very likely cm-level positioning available to millions if not billions of people in cheap handheld devices, the issues of professional education and attention to detail are more important than they have ever been. While it would be really nice if everyone who picked up a GNSS receiver had an advanced degree in geodesy, obviously that is not only unrealistic, it’s senseless. What does need to happen is a comment that I’ve made in hundreds of seminars on these topics — those in professions and disciplines where high-accuracy coordinates are important should know enough to qualify for the Junior Geodesist Secret Decoder Ring!
There are efforts in the works at this time that may bring us a step closer to making this a reality. The American Association for Geodetic Surveying (AAGS) is working on a geodetic surveying certification initiative in collaboration with the National Society of Professional Surveyors (NSPS). This effort will be aimed at anyone who is inclined to collect, manage, distribute and/or utilize the increasing amounts of high-quality positional information.
Watch this space for more details next time.
David Doyle joined the National Geodetic Survey in 1972, and held the position of chief geodetic surveyor for 12 years before his retirement in January 2013. He was responsible for the development, technical design and management of plans and programs that enhanced the United States National Spatial Reference System. During his career with NGS, his experiences included all phases of geodetic triangulation, astronomic positioning, leveling, GPS data collection, data analysis, datum transformations, network adjustments, data publication and outreach in the form of seminars, workshops and webinars. His efforts also included extensive activities to direct and coordinate the modernization of national geodetic reference frames in countries in Africa, Central, Caribbean and South America, Eastern Europe and the Pacific.
Doyle is a past president of the American Association for Geodetic Surveying and a Fellow member of the American Congress on Surveying and Mapping. He has served on the U.S. delegation to the International Federation of Surveyors and is an active member of the District of Columbia, Maryland and Virginia professional surveyors associations. Doyle now operates Base 9 Geodetic Consulting Services.
The unmanned drone RQ-4 Global Hawk in flight in 2007 (Image credit: U.S. Air Force photo by Bobbi Zapka)
By Art Kalinski, GISP
For more than a decade, the military has been struggling with cataloging and retrieving its huge libraries of full motion video (FMV). The video, captured by both manned and unmanned aircraft, rapidly reached unmanageable levels. If you have ever tried to organize vacation photos after returning home from a long trip, you know that it’s easy to lose track of where each photo was taken. Date/time stamps help, but the effort is still difficult if your vacation took you to numerous locations.
The manned U.S. Air Force Beech King Air 350 and 350ER MC-12W Project Liberty Aircraft are designed for intelligence, surveillance and reconnaissance (ISR).
Now imagine trying to retrieve several critical minutes out of thousands of hours of video of barren land or repetitive-looking villages, and you get a sense of the magnitude of the problem. Without some way to catalog the video, critical details can be lost, because finding the right video clip becomes impossible in a reasonable period of time. Everyone agreed that the solution to the problem is to index the video clips by date/time and location, preferably with an exact georeferenced footprint. This is now possible with tools from Esri, Hexagon Geospatial and others.
Several years ago, at the USGIF GEOINT Tech Days, Sarnoff (SRI) demonstrated a system that pinned aerial video to its geographic footprint and maintained that registration despite the movement of the aircraft. It was an achievement that impressed everyone in the audience. I changed jobs soon after that conference and lost track of developments in the FMV field. At the February Esri Federal Users Conference, I was thrilled to see Jack Dangermond briefly demonstrate the same kind of FMV georeferencing capability in ArcGIS 10.2 during his opening plenary session. I learned that Esri developed the capability in 2013, and later learned that Intergraph (Hexagon Geospatial) also developed a similar capability in 2010.
MISB: The Critical Improvement
The key technical development that made this possible was NGA’s creation of “Motion Imagery Standards” and the Motion Imagery Standards Board (MISB). The MISB developed standards for a consistent way to capture and record telemetry data during the video capture as metadata that becomes part of the video stream. This “Open Standard” metadata includes information such as the accurate xyz location of the aircraft, attitude, tilt, camera angle, and camera characteristics. This information travels with the individual video frames and permits the GIS/viewing software to perform the georeferencing.
This process is very similar to the oblique imagery capture system used for years by Pictometry, which at 20 FPS was technically FMV. MISB like Pictometry requires accurate GPS and IMU data to continuously capture and record the metadata. The MISB also gets involved with video compression standards such as the newer H.264 used on Blu-Ray discs and streaming video. H.264 has, for the most part, replaced MPEG2 and older MPEG4 as the video compression standard of choice. Much of the video captured by low-end small UAVs is just a simple video stream with no MISB telemetry data. However, I’m sure that lower prices, increased capability and smaller size of sensors will fix that with time. Sorry, no one has yet figured a way to “hack in” the metadata for legacy video captured without the MISB telemetry data. The one exception is those videos that contain usable telemetry data that was burned into the video and can be read with OCR. It might be possible to insert that information as MISB-compliant data.
The ArcGIS Full Motion Video Add-In
The ArcGIS Full Motion Video 1.2 Add-In (for ArcGIS 10.1, 10.2 and 10.3) is a free tool for ArcGIS users. It permits users to play georeferenced live or previously recorded video files in the map view. The screen capture below shows several features of this tool and is from an online video.
The re-sizeable smaller window displays the video as it runs. The map view shows a changing footprint (green trapezoid) of the video as the aircraft flies over the site. The short green line shows the flight path of the aircraft. Demonstrating the interconnectivity of the two data sets, the user in the demo video drew a light blue polygon on the map view. Note that the Esri Intersect function re-projects and displays the same polygon correctly in the video view.
Frames from the video can be extracted as single georeferenced images or groups of images and stored as a mosaic dataset. Playback of time-stamped video data can be synchronized with other time-enabled data and played together on the map. Features can be digitized directly on the video player and will appear on the map and vice versa.
The extension supports playback and management of multiple simultaneous video feeds. The Add-In also allows you to record the sensor, frame center, and footprint data in a geodatabase so the Bookmark Manager can perform searches for bookmarked video scenes. For more information regarding the Esri FMV tool, visit this ArcGiS site.
Hexagon Geospatial GeoMedia Motion Video Analyst Professional
Another robust FMV system that takes advantage of MISB telemetry is GeoMedia Motion Video Analyst Professional (MVA) from Hexagon Geospatial. The Hexagon Geospatial system includes tools to catalog videos as geospatial features with attributes extracted from the metadata, and has some elegant graphic selection tools that help an operator search and retrieve needed video clips.
In the screen capture below, you can see the map view with the geo-registered color video overlaid on the black-and-white ortho base image. As the video plays, the georeferenced video footprint continuously moves to the correct location as the tracking graphics in red show the position of the aircraft. The transparency of the video can be adjusted so an operator can compare features between the base image and the video and digitize directly in the map window.
MVA also includes a full set of tools for placing clipmarks as geospatial features with attributes and linked to the cataloged video, extracting snapshots and videos clips, on-the-fly enhancements, stabilization, registration and more. The system also facilitates rapid report generation so as an operator searches and plays appropriate video clips, the same operator or a partner can rapidly generate reports as documents or PowerPoint presentations in minutes. Another feature of the system is a “de-hazing” tool that removes a surprising amount of haze or smoke.
See a very good video overview of GeoMedia Motion Video Analyst Professional on this Hexagon Geospatial page. Like the Esri video, both are far better at explaining the capabilities than I can in this short column.
Other defense contractors are taking advantage of the MISB metadata, so check with your provider. Although these systems found their first home with military analysts, the Esri and Hexagon Geospatial reps indicated that many other users are finding the capability valuable in their work. Emergency operations centers come to mind first, but more mundane uses include rail and utility property management, the news media and video used in court proceedings. So, if you shoot lots of aerial video and need to catalog and retrieve video clips quickly, consider using MISB in your capture process.
The Institute of Navigation’s GNSS Software Defined Radio Metadata Standard working group is seeking public comment on the standard through Dec. 31.
