High-resolution imagery geolocated by the sixth-generation Digital Sensor System (DSS) after Hurricane Ida. (Photo: NOAA)
Applanix, a division of Trimble, has been working with the National Oceanographic and Atmospheric Administration (NOAA) since the early 2000s to develop their response for emergency and coastal mapping activities. We discussed this collaboration with Joe Hutton, the company’s director of inertial technology, land and airborne products.
How has Applanix collaborated with NOAA regarding emergency response and coastal mapping?
Early on, we worked with them to develop a solution that allowed them to get out in the field and produce high accuracy map products with minimal touching of the data. In mid-2021, we delivered the next generation of this solution, or the DSS version six, which represents the culmination of everything learned over the years about how to produce imagery for emergency response, in terms of the types of collection, the types of imagery, and how to get it into first responders’ hands as quickly as possible.
At the heart of the system is our direct georeferencing technology. It’s a solution that allows us to assign the geographic location of every pixel of the digital imagery collected in the air. As soon as you land, you have the coordinates of every pixel, which means that you have a map that NOAA then pushes to the cloud for first responders to use in their emergency response efforts.
The collaboration consisted of Applanix working with Lead’Air to manufacture the next generation system that meets NOAA’s latest requirements. That’s what we delivered in 2021. Weeks after delivery, NOAA was called to respond to the hurricanes. They flew the new system with great success and were able to use it for their response.
What is your perspective on ground control points (GCPs) vs. direct georeferencing?
It is impossible to place GCPs in an emergency response when you cannot get on the ground. People who say they need GCPs do not really understand direct georeferencing. We’re having this debate even after 20 years of proving this technology. The NOAA system does not use GCPs and the map products are at centimeter level accuracy.
We use Trimble’s RTX technology, which enables centimeter-level GNSS positioning without base stations, which is important when the CORS or local RTN is unreliable due to a disaster. We have high accuracy inertial systems that get us the high accuracy orientation, so that we can go directly to ortho photos and ortho mosaics without running any triangulation or using GCPs in that process. That is a standard process these days. GCPs are only there for quality control if you want to deliver a final map product.
Did NOAA fly the mission with its own aircraft?
Yes, these are NOAA’s King Air or Twin Otter aircraft. The King Air aircraft is specifically outfitted for these types of emergency response and coastal mapping activities. The DSS system gets installed into the airplane and gets calibrated in terms of checking the system out for accuracy. Then it’s ready to fly the response. In the air, they collect the imagery over a flight path of interest to them. Then, it’s developed from raw imagery into JPEGs in the aircraft, and all the georeferencing data is logged with that imagery so that as soon as they land they can push a button and start to reference the JPEG imagery and push it to the cloud.
What are the components of your system?
What makes this system so unique is that it encompasses all the lessons learned over the years in terms of what NOAA needs to optimize for both their coastal mapping and their emergency response. It incorporates two pairs of color and near-infrared Phase One cameras that are configured in an oblique format with some overlap, forming a bowtie footprint on the ground.
You have 100% overlap of the color with the near-infrared and it’s on a high-performance stabilized mount that keeps everything perfectly level. The mount also has a special feature that enables the operators to rotate the cameras to go into nadir mode, mostly for traditional coastal mapping that requires stereo imagery. We were able to incorporate into a single system the requirements for both emergency response—where you want large coverage and obliqueness to look for damage—and nadir for coastal mapping.
Lead’Air built the sensor for you, on your specs, correct?
Yes, that’s correct. We’ve worked with Lead’Air for probably 20 years on flight management system (FMS) technology. They also have an amazing capability to build stabilized mounts and hardware systems. So, we decided to work together. We contracted them to implement some of their innovative hardware in this new design for us to deliver to NOAA. We contracted them to do all the manufacturing of the design and delivery to NOAA.
One of the quite innovative things that they did was to develop a new flight management capability that allows NOAA to fly ad hoc along highways or rivers, looking for damage. Traditionally, for aerial imagery you have to pre-flight plan trajectories. They designed an FMS that enables a pilot to fly a road or a river looking for damage without worrying about traditional block collections as with a more traditional FMS. So that feature further increases productivity. If you look at the most recent imagery at www.storms.ngs.noaa.gov you will see that it looks like spaghetti, not like blocks. That’s because they are following the roads and the rivers looking for specific damage.
Does the post-processing use your software?
Yes, it uses the POSPac MMS post processing software with POSPac Trimble Post-processed CenterPoint RTX correction service, allowing us to get that centimeter-level position accuracy, anywhere in the world with just an internet connection. You don’t have to worry about having a local base station—which, of course, if you’re in an emergency response situation, might not be there anyway. So, this is a very powerful way of getting global centimeter-level accuracy in real time, without having to worry about the ground-based GNSS infrastructure, that is, the local real-time network, that’s on the ground.
If you don’t have internet access, you can ship that data to the nearest place that does, right?
You could, however NOAA simply flies to wherever there is access. What takes the longest is to develop the imagery from the raw format to the JPEG format, because these are such large images. Doing that in the air saves an enormous amount of time. You have these JPEG-ready images that are compressed and can go right into the georeferencing process and make it really, really fast.
That’s a matter of computing power and smart software. What else did Lead’Air contribute?
This very efficient, fast image development process in the aircraft.
It sounds like it was a very integrated process between Applanix and Lead’Air. So, NOAA had the instrument mounted on their aircraft, their pilots did the flying, and then you processed the data?
No, NOAA’s team processes all the data. We just deliver the hardware and the software. They created the workflow software to push the data to their cloud environment.
NOAA uses this data to produce maps of the damage and highlight different situations and hazards?
Yeah. When these hurricanes go through, the first questions people have are “Where’s the damage? Are these roads passable? Did my house survive?” If you are doing response, you need to get teams in there. First, however, you need to know whether the roads are passable, so that you will not waste time going down a road that is not. So, the first thing they do is go up in the air and survey the main roads to push the imagery back, so that people can assess whether the roads are passable. Then they start to look for specific areas of damaged infrastructure, to triage where to put their resources. Then they ask “How do we manage disaster recovery?”
What lessons did you learn?
We are still learning about the power of the system, because these are Phase One 150 megapixel color cameras. It is such a powerful combination of sensors that they’re starting to look at different information they can get out of these things. They’re still learning new lessons in terms of what information can be useful for both the emergency response and the coastal mapping.
Ultimately, we’ll go to full ortho maps in the aircraft. That’s just going to be a matter of computational power. The holy grail would be to produce an orthophoto in the aircraft and radio it down to the ground in real time. Nothing prevents you from doing that now other than computational power and bandwidth. It’s not practical yet, but it will probably get there.
Do you have collaborations like the one with NOAA with any other major U.S. agencies?
We’ve worked extensively with NASA over the years. For example, we have worked with them on the ice bridge project. That is where they survey ice at both poles to measure its thickness and how global warming is affecting it. They use our system on that to do the georeferencing. We also work extensively with other branches of NOAA for their shoreline mapping from their ships. We have worked with them over the years to provide the georeferencing solution for the multibeam echo sounders to produce their nautical charts.
Surveyors used ComNav equipment to construct a hospital in Burkina Faso. (Photo: ComNav)
Line of sight to GNSS satellites is sometimes obscured by buildings and trees, which also cause multipath, as does nearby water. These conditions require an RTK receiver with multipath mitigation. Often, surveying must occur on property corners or on uneven ground, where it is hard to place surveying equipment. For these reasons, reliability and accuracy are essential, especially in harsh environments. Ground control points require 1-2mm accuracy and topo surveys 1-2cm accuracy. Surveying for AEC also requires software that processes digital files.
ComNav has focused on GNSS core technology innovation and applications for 10 years. The Quantum III technology includes algorithms to suppress multipath and supports all GNSS constellations, allowing the users to acquire and keep RTK centimeter accuracy even in harsh environments. The built-in tilt IMU will help where the exact location to be surveyed is hard to reach. For example, the T300 Plus and N Series GNSS receivers support a maximum pole tilt of 60° and keep the compensation accuracy within 2.5cm, making the field work more efficient, convenient and reliable.
With the Survey Master software’s stake-out points, users can import DXF or DWG files directly and the software can stake out the point, line and surface in CAD.
In April 2021, the government of Burkina Faso used ComNav GNSS T300Plus to provide ground control points survey for the construction of a hospital.
The land security and topographic surveying were completed within only six days, less than half the time that had been scheduled for those tasks. This greatly expedited the construction of the hospital and helped with the fight against infectious diseases, including COVID-19.
Kinematic Ground Control point for UAV photogrammetry: A dynamic duo of UAV and mobile van combine to deliver the accuracy of conventional methods with only 2+2 ground control points at the ends of the corridor.
By Ismael Colomina, Pere Molina and Roberto da Silva Ruy
A Brazilian and a Spanish company, ENGEMAP and GeoNumerics respectively, have finalized the accuracy evaluation of a mission conducted with the latter’s mapKITE technology on a Brazilian motorway in 2018.
The goal of the evaluation was to confirm the advantages of the mapKITE method and its kinematic ground control point (KGCP) concept over conventional corridor mapping methods.
The mapKITE and the conventional method delivered comparable accuracy results — the difference being that the latter requires a dense set of surveyed ground control points (GCPs) while mapKITE does the job with almost no GCPs.
For this purpose, a 4-kilometer segment of the Rodovia Raposo Tavares in São Paulo state was populated with a set of 37 evenly distributed, signalized, accurately surveyed ground points. The set was divided into two subsets of 23 GCPs and 14 ground check points (GChPs) — the ground truth — respectively. The 4-km road segment was also covered by 189 drone images and their corresponding 189 KGCPs. The image set was processed as a conventional aerial corridor block:
with the integrated sensor orientation (ISO) method in a 23 GCP + 14 GChP configuration, and
as a mapKITE aerial corridor block in a 4 GCP + 14 GChP + 189 KGCP configuration.
The two processes produced similar accuracy results: mean (μ), empirical standard deviation (σ) and root mean square (rms) error of the photogrammetric determination of the horizontal (EN) and vertical (h) coordinates of the GChPs against the ground truth. (All units are stated in millimeters.)
The mapKITE configuration uses only four GCPs (two at each end of the road segment) in contrast to the 23 GCPs of the conventional method. Nominal flying height of the drone was 120 meters above ground, producing an average ground sampling distance (GSD) of 2.3 cm. Forward image overlap was 80% resulting in a base-to-height ratio of 0.157.
MapKITE is a GeoNumerics patented method for 3-dimensional corridor mapping that combines the two latest geodata acquisition methods, terrestrial mobile mapping and aerial drone-based mapping. MapKITE is a tandem terrestrial-aerial mapping method and system composed of:
a terrestrial mobile mapping system (land vehicle and sensors) carrying
an optical metric target on its roof;
a drone aerial mapping system; and
a real-time virtual tether and post-mission software.
In a mapKITE mission, the drone follows the land vehicle, and thus the vehicle target becomes a kinematic ground control point visible and measurable on each image. It is a high-accuracy, high-resolution Earth observation method. MapKITE combines the advantages of mobile land-based encompassing images and 3D point clouds. MapKITE combines the advantages of mobile land-based (manned) and aerial drone (unmanned) mapping systems.
GeoNumerics (Castelldefels, Spain) is a research and development company specializing in geomatics and accurate navigation.
ENGEMAP (Assis, Sao Paolo, Brazil) is one of the largest and oldest mapping companies in Brazil. It has more than 100 employees, three aircraft, two mapping land vehicles, a number of rotary- and fixed-wing drones and a record of accomplished mapping and cadastral projects. ENGEMAP is officially authorized by the Brazilian Ministry of Defence (MD) and the Brazilian Department of Airspace Control (DECEA) to conduct mapKITE commercial flights in Brazil.
MANUFACTURERS
The mapKite campaign was conducted with a Sensormap SMM terrestrial mobile mapping system and a UAVision UX Spyro drone equipped with a NovAtel OEM2 GNSS dual-frequency receiver with a Maxtena antenna and a Sony α7R camera with a 25-mm camera constant lens. The INS/GNSS system in the Terrestrial Vehicle was a Span-CPT (Novatel) including dual-frequency antenna and DMI wheel sensor.
ISMAEL COLOMINA is chief executive and chief scientist at GeoNumerics. He has a Ph.D. in mathematics from the University of Barcelona.
PERE MOLINA is advanced applications program manager at GeoNumerics. He holds a master’s degree in mathematics from the University of Barcelona and a master’s in photogrammetry and remote sensing from the Institute of Geomatics, Catalonia.
ROBERTO DA SILVA RUY is technical manager at ENGEMAP. He has a Ph.D. from the Universidade Estadual Paulista.
Datumate has released DatuSurvey version 5.1 for both Professional and Enterprise editions of the software. DatuSurvey (formerly DatuGram 3D) turns drone- and camera-based images to accurate, georeferenced 2D maps and 3D models, which saves the need for expensive and risky field work and expedites deliveries, according to Datumate.
DatuSurvey Professional V5.1 now also includes:
Ground Control Points Hints – Once the model is built with the minimal requirement of 3 GCP’s on two images each, the system will start showing hints for selected GCP on all images it is not marked in. This will make the GCP marking easier and faster.
Differentiating Clusters in Map View – Different clusters are now shown in different colors in the map view.
DatuSurvey Enterprise V5.1 now also includes:
Dense Point Cloud Generation Quality – Dense Point Cloud may now be generated at four different density levels as specified by the user.
Mesh and Texture Support – Dense Point Cloud may now be generated with mesh or with textured mesh. Mesh and texture may be exported to OBJ format.
True Orthophoto Export Quality – Orthophoto may now be generated at four different resolutions.
Visualization Viewer Improvement – 3D Viewer is able to handle up to 100 million points. Thus, viewing an excellent quality model with mesh and texture.
Volume Calculation Improvement – Volume calculation was improved to allow definition of stockpiles right on the dense point cloud, including physical and base surfaces. The definition process is now faster and easier, and the volume calculation of more precise.
Ground Control Points Hints – Once the model is built with the minimal requirement of three GCP’s on two images each, the system will start showing hints for selected GCP on all images it is not marked in. This will make the GCP marking easier and faster.
Differentiating Clusters in Map View – Different clusters are now shown in different colors in the map view.
Aeropoints are desgined for for companies across the industrial sector — including mining, construction, quarries and landfills.
Propeller Aero has introduced AeroPoints — smart ground-control points designed to make it easy to capture surveyaccurate mapping using drones.
The patent-pending technology provides a simple solution to a major roadblock to widespread commercial drone adoption: accuracy.
Typical ground control requires establishing precise geolocation position using surveying equipment, and then securing a visible ground marker exactly on the pre-marked GPS point.
AeroPoints are portable ground-control markers, visible from the air and capable of quickly capturing their own positions down to 2-centimeter absolute accuracy.
AeroPoints work with any camera or drone, and integrate seamlessly with Propeller’s cloud-based data platform and processing engine (see above story). They’re solar-powered, durable and weather resistant, and they don’t require any onsite connection.
To use AeroPoints, customers simply lay them down, fly their drone, and then pick them up again. They automatically connect to a wireless or mobile hotspot when back in range to upload captured positional data — and precision georeferencing is done.
Four point clouds, nonregistered, of georeferenced images from four UAV flights.
By Christian Eling, Lasse Klingbeil, Markus Wieland, Erik Heinz and Heiner Kuhlmann
Direct georeferencing with onboard sensors is less time-consuming for data processing than indirect georeferencing using ground control points, and can supply real-time navigation capability to a UAV. This is very useful for surveying, precision farming or infrastructure inspection. An onboard system for position and attitude determination of lightweight UAVs weighs 240 grams and produces position accuracies better than 5 centimeters and attitude accuracies better than 1 degree.
Data acquisition from mobile platforms has become established in many applications recently, particularly using unmanned aerial systems (UASs). Unlike other mobile platforms, unmanned aerial vehicles (UAVs) can overfly inaccessible and also dangerous areas. Furthermore, they can get very close to objects to collect high-resolution data with low-resolution sensors, and they enable approach from all viewing directions without physical contact. UAVs now see use in precision farming for phenotyping or plant monitoring, and in infrastructure inspection and surveying.
Data acquisition from mobile platforms has become established in many applications recently, particularly using unmanned aerial systems (UASs). Unlike other mobile platforms, unmanned aerial vehicles (UAVs) can overfly inaccessible and also dangerous areas. Furthermore, they can get very close to objects to collect high-resolution data with low-resolution sensors, and they enable approach from all viewing directions without physical contact. UAVs now see use in precision farming for phenotyping or plant monitoring, and in infrastructure inspection and surveying.
This article addresses lightweight UAV use for mobile mapping and uses the term micro aerial vehicle (MAV) throughout. MAVs can generally be characterized as having a weight limit of 5 kilograms and a size limit of 1.5 meters.
We focus on the development of a real-time capable, direct georeferencing system for MAVs, since spatial and time restrictions often exclude the possibility of deploying ground control points for an indirect georeferencing. The demand for the real-time capability results from the aim to also use the georeferencing for autonomous navigation of the MAV and to enable a precise time synchronization of the onboard sensors. Furthermore, a real-time direct georeferencing also offers the opportunity to process collected mapping data during flight.
Mapping on demand. The goal of this research project, funded by the Deutsche Forschungsgemeinschaft (DFG), is to develop an MAV that can identify and measure inaccessible three-dimensional objects by use of visual information. A major challenge within this project comes with the term “on demand.” This means that apart from the classical mapping part, where 3D information is extracted from aerial images, the MAV is intended to fly fully autonomously on the basis of a high-level user inquiry. During the flight, obstacles must be detected and avoided. To extract semantic information that can be used to refine the trajectory planning, the mapping data has to be processed in real time. When the georeferencing information is used as initial values for the bundle adjustment, the image processing can be significantly accelerated.
Figure 1 shows the current MAV platform developed in this project. We customized an MAV kit to a coaxial rotor configuration, replaced the centerplates with more stable carbon-fibre plates to stabilize the system, and installed the direct georeferencing and the mapping sensors. The two stereo camera pairs, pointing forward and backward, act as an additional sensory input for the position and attitude determination; the 5M-pixel industrial camera with global shutter is the actual mapping sensor. The PC board is used for onboard image processing, flight planning and machine control; the Wi-Fi module enables a connection to a ground station.
Figure 1. The MAV with mapping and georeferencing sensors, developed for the research project Mapping on Demand.
Although the direct georeferencing system must be small and lightweight, accuracy requirements for its position and attitude determination are high. Generally, these accuracy requirements are different for the machine control, navigation and mapping purposes.
In our project, the MAV is intended to maintain a safety distance of about 0.5 meter to obstacles. Hence, a position accuracy of 0.1 meter is sufficient for the navigation. The absolute attitude accuracy should be in the range of 1 to 5 degrees. For machine control, relative information is more important, and for this the accuracies should be slightly higher.
For mapping purposes, the positions and attitudes have to be known better, since the absolute georeference of the final product (for example, a high-resolution 3D model of a building) is based on the positions and attitudes from the direct georeferencing system. Therefore, the position accuracy should be in the range of 1–3 cm and the attitude accuracy should be better than 1 degree. The relative accuracy of the exterior camera orientation can be improved by a photogrammetric bundle adjustment, but systematic georeferencing errors should be avoided.
To summarize:
The weight of the system has to be less than 500 grams (g), to be applicable on MAVs.
Especially for the control and navigation, the system has to be real-time capable.
All sensors have to be synchronized and outages of single sensors should be bridgeable by other sensors.
The system is intended to provide accurate positions (σpos < 5 cm) and attitudes (σatt < 1 deg) during flights.
The integration of data from additional sensors, such as cameras, should be possible.
The ability to include additional sensors to the system was, apart from the size and the weight constraint, the main reason for developing a proprietary system instead of using a commercial unit with similar capabilities.
Direct Georefencing
The current version of the system weighs 240 g without GPS antennas (see figure 2). To reduce weight, the antennas were dismantled, reducing their weight from 350 g to 100 g. However, since the antenna reference point got lost in this process, the antennas had to be recalibrated in an anechoic chamber for further use. By comparison to the original antennas, the dismantling led to significant changes in the phase center offsets (circa 4 cm in the Up, < 1 mm in the North and East component) and in the phase center variations (< 5 mm) of the antennas.
Figure 2. The direct georeferencing system.
Figure 3 shows a flow chart of the direct georeferencing system with the sensors and the main calculation steps. The system consists of a dual-frequency GPS receiver, a single-frequency GPS receiver, an inertial measurement unit (IMU) and a magnetometer. The dual-frequency receiver is the main positioning device. Together with the GPS raw data from the master station (carrier phases ϕM, pseudoranges PM), which is transmitted via a radio module, the data of the dual-frequency receiver (ϕR, PR) is used for an RTK positioning, leading to centimeter position accuracies.
Figure 3. Flowchart of the direct georeferencing system.
In collaboration with the data of the single-frequency receiver (ϕB, PB), the data of the dual-frequency receiver is also used for GPS attitude determination. The corresponding GPS antennas of these two receivers form a short baseline (baseline length = 92 cm) on the MAV. The determination of the baseline vector in an e-frame (Earth-fixed) enables yaw and the pitch-angle determination.
The tactical-grade micro-electro-mechanical (MEMS) IMU, which includes three-axes gyroscopes, accelerometers and magnetometers, provides angular rates (ω), accelerations (a) and magnetic field observations (h) with high rates (100 Hz) for position and attitude determination. To be unaffected by the electric currents as much as possible, an additional magnetometer is placed on the outer end of one of the rotor-free MAV arms.
The direct georeferencing system further consists of a processing unit, which is a reconfigurable IO board, including a field programmable gate array (FPGA) and a 400-MHz processor. In this combination, the FPGA is used for fast parallel communication with the sensors. Afterwards, the preprocessed sensor data are provided to the 400-MHz processor via direct memory accesses, avoiding delays and supporting the system’s real-time capabilities. Finally, the actual position and attitude determination is carried out on the 400-MHz processor.
Methodologies
All position and attitude determination algorithms running on the system were developed in-house. Generally, the integration of these steps could be realized in one tightly coupled approach. Nevertheless, in the current implementation, we decided to separate the different raw data calculation steps, and we only use interactions at the level of parameters. This approach has the advantage that the integration is more reliable and more practical in the real-time programming.
GPS/IMU integration. In this calculation step, all available sensory input is fused to determine the best position and attitude of the system that is currently available. The GPS and the IMU measurements complement each other well, since the IMU provides short-term stable high-rate (100 Hz) data, and the GPS provides long-term stable low-rate (10 Hz) data.
The GPS/IMU integration can be separated into the strapdown algorithm (SDA) and the Kalman filter update. In the SDA, the high-dynamic movement of the system is determined integrating the angular rates and the accelerations of the MEMS IMU in real time. Because the SDA drifts over time, the long-term stable measurements of the magnetometer and the GPS receivers are needed to correct and bound the drift of the inertial sensor integration, which is realized in an error state Kalman filter.
In the GPS/IMU integration algorithms, the navigation equations of the body frame (b-frame) are expressed in an e-frame. Therefore, the full state vector x includes the position xep and the velocity vep, represented in the e-frame. For the attitude representation a quaternion q is used. Finally, the accelerometer bias bba and the gyro bias bbω are also estimated:
The observations in the measurement model are:
the RTK GPS position xea of the dual-frequency RTK GPS antenna reference point, expressed in the e-frame,
the GPS attitude baseline vector Δxeb, expressed in the e-frame,
the magnetic field vector hb, expressed in the b-frame.
Because the reference point of the RTK GPS antenna is not identical to the system reference point, a lever arm between the system and the antenna reference point must be regarded in the measurement model of the RTK GPS positions. From calibration measurements, the coordinates of the lever arm are precisely known in the b-frame.
In the SDA, a coupling between the accelerations, measured by the IMU, and the positions, measured by the RTK GPS, exists. Due to this coupling the yaw angle can be observed, but only in the presence of horizontal accelerations.
To determine an accurate and reliable yaw angle for every motion behavior, the short GPS baseline is realized on the MAV. A significant challenge in processing this baseline is the ambiguity resolution, because only single-frequency GPS observations can be used. Empirical tests have shown that the ambiguity resolution of a single-frequency GPS baseline generally takes several minutes. Among other strategies, we use the additional information from a magnetometer to improve the ambiguity resolution and to actually enable an instantaneous ambiguity fixing during kinematic applications.
Ferromagnetic material on the UAV and high electric currents of the rotors create significant disturbances of the magnetometer during flight. While the influence of the material can be compensated by calibration procedures, the influence of the dynamically changing electric currents are more challenging. To minimize them, the magnetometer is placed at the outer end of a rotor-free arm of the MAV. Also, the measurement model is arranged so that magnetic field observations only have an impact on the yaw determination in our algorithms.
RTK GPS Positioning. RTK GPS positions are calculated in real time with a rate of 10 Hz. These RTK algorithms are in-house developed, although commercial and open-source solutions are available. The main reasons for developing custom software are the following:
Integration of other sensors and/or solutions is possible, to improve ambiguity resolution and cycle-slip detection.
In commercial software, there is generally no access to the source code.
In the development of a real-time capable system, the software must meet the requirements of the operating system running on the real-time processing unit.
Generally, the RTK GPS algorithm complies with a single baseline determination (one master, one rover), where the master station remains ground-stationary and the rover is onboard the MAV.
To resolve the ambiguities and finally to determine the RTK GPS positions, the parameter estimation is performed in three steps: float solution, integer ambiguity estimation and fixed solution.
The float solution is realized in an extended Kalman filter (EKF). Beside the rover position, represented in the e-frame, the EKF state vector xSD also contains single-difference (SD) ambiguities N j on the GPS L1 and the GPS L2 frequencies. The reason for estimating SD instead of double-difference (DD) ambiguities is to avoid the hand-over problem that would arise for DD ambiguities, when the reference satellite changes.
To allow for an instantaneous ambiguity resolution, the observation vector l consists of DD carrier phases Φjkrm and DD pseudoranges Pjkrm on the GPS L1 and the GPS L2 frequencies.
In the current implementation, a random walk model is assumed as a dynamic model of the MAV in the EKF. Even if this is a simple model, it complies with the movement of the vehicle, when the process noise is chosen appropriately.
The float solution procedure provides real-valued ambiguities and their covariance matrix. These ambiguities now must be fixed to correct integer values, to fully exploit the high accuracy of the carrier phase observables. We applied the MLAMBDA method for integer ambiguity estimation.
Finally, a decision must be made whether or not the result of the integer ambiguity estimation can be accepted. This is done by the simple ratio test. With the ambiguities fixed, the final rover position xae is estimated with cm accuracies.
Usually, the time to fix the ambiguities with the algorithm takes a few epochs, but often the ambiguities can be fixed instantaneously. Once ambiguity resolution has been successful, the ambiguities can be held fixed, as long as no cycle slip or loss of lock of GPS signals occur.
Due to the GPS/IMU integration, we have a precise prediction of the RTK GPS positions between two epochs. Thus, the integration of the inertial sensor readings enables us to detect and also repair cycle slips very reliably.
The observations of the master receiver must be transmitted via radio to the direct georeferencing system. In practice, this data transmission can only be realized with a rate of 1 Hz. To be less dependent on this potentially unreliable master data transmission and the lower sampling rate, simulated master observations are used for RTK GPS position determination. Hence, in the actual processing, the true master observations are only used to update the simulation errors in the master task (figure 4), which have to be applied to correct the simulation results in the rover task.
Figure 4. Task scheduling of the RTK GPS algorithms.
GPS attitude determination. The GPS baseline is determined at 1 Hz. In contrast to the RTK GPS positioning, both antennas of the attitude baseline are mounted on the MAV, so that the complete baseline is moving. Furthermore, the baseline length is constant and known from calibration measurements. The GPS attitude determination also consists of the three steps: float solution, integer ambiguity estimation and fixed solution.
The float solution is also based on an EKF where the single-frequency SD ambiguities N j of the attitude baseline are estimated. Further parameters in the state vector are the baseline parameters and the first deviation of the baseline parameters.
As observations DD carrier phases ΦjkAB and DD pseudoranges PjkAB on the GPS L1 frequency are used. To improve the ambiguity resolution, the attitude from the GPS/IMU integration is added to the observation vector, by transforming the known b-frame baseline parameters into the e-frame. Finally, also the known baseline length can be added as a constraint to the observation vector.
In the integer ambiguity estimation, we apply the MLAMBDA method again. Due to the prior information about the attitude of the baseline, the float ambiguities can already be estimated with high accuracies in the float solution. If the ambiguities could not be fixed with the MLAMBDA method, we consider the 10 best solutions for further processing. Unreliable ambiguity parameters are eliminated in a random order, and the MLAMBDA method is applied again. Afterwards we use the ambiguity function method and the known baseline length to exclude false candidates of the 10 best solutions.
If only one solution remains, the ambiguities can be fixed to integer values. Tests have shown that this approach leads to an instantaneous ambiguity resolution success rate of about 95 percent.
Similar to the RTK GPS positioning, the IMU readings are also used to detect cycle slips for the attitude baseline determination, when the ambiguities have been fixed successfully. With ambiguities fixed, the baseline parameters can be determined with millimeter to centimeter accuracies. This leads to yaw angle accuracies in the range of 0.2–0.5 degrees, when the attitude baseline has a length of 92 cm.
Applications and Results
As mentioned, one goal of Mapping on Demand is 3D reconstruction from visual information. The opening image shows such results. During four flights. images were collected with a sampling rate of 1 Hz, and the position and the attitude of the camera was determined in real time using the direct georeferencing system. A bundle adjustment was processed using these positions and attitudes as initial values. Afterwards, dense point clouds could be generated from the oriented images using an open-source software package (PMVS). Due to georeferencing of the collected images, the point clouds are also georeferenced. The image shows results of four flights in one scene, to demonstrate consistency of the georeferencing.
Agriculture. In figure 5, georeferenced images were taken during a flight over a wheat field. The same process was repeated after two weeks. The difference of the respective point clouds, which were determined using the software Photoscan by the company Agisoft, reveals the plant growth at an interval of two weeks. These results show that the determination of plant growth rates, which usually result from time-consuming field work, can be done easily and with high resolution using MAVs. With the use of a direct georeferencing system, this process becomes even more efficient because the deployment of ground control points can be omitted.
Figure 5. Orthophoto of a wheat field (left) and the difference of the vegetation height, determined from the results of two MAV flights at an interval of two weeks (right).
Portable laser scanning system. The small and lightweight design of the direct georeferencing system offers several other opportunities for various applications. One example is the use of the direct georeferencing system in combination with a small, lightweight and low-cost laser scanner.
Terrestrial laser scanning has become an established technology for 3D data acquisition in surveying and mapping because laser scanners provide high-resolution data with high accuracies at high speed. However, for measurement of a complex scene, the laser scanner generally has to be moved to different viewpoints, and all measured scenes have to be registered and georeferenced, a significant increased effort. In contrast, with a directly georeferenced kinematic laser scanning system, complex scenes can be measured with little effort.
Figure 6 shows a portable laser scanning system we developed for kinematic laser scanning. It combines the direct georeferencing system with a low-cost, lightweight 2D time-of-flight laser scanner. Time synchronization and the point cloud calculation are directly realized on this unit.
Figure 6. A directly georeferenced portable laser scanning system for kinematic 3D mapping.
Figure 7 shows differences between a directly georeferenced point cloud, measured by the portable laser scanning system, and a terrestrial laser scanning point cloud, which was indirectly georeferenced using ground control points. Although there are some systematic errors visible, the differences are mostly less than 7.5 cm. The larger differences in the foreground (red) are a result of growing vegetation in the period between both scans. The systematic errors result from the system calibration between the laser scanner and the direct georeferencing system. We are working to improve these calibration methods.
Figure 7. Difference between the results of the directly georeferenced portable laser scanning system and the results of a terrestrial laser scan, which act as reference solution here.
Manufacturers
The MAV is based on a MikroKopter OktoXL assembly kit of HiSystems GmbH. It uses NavXperience 3G+C GPS antennas. The system consists of a dual-frequency NovAtel OEM 615 GPS receiver, a single-frequency u-blox LEA6T receiver, an Analog Devices ADIS 16488 IMU, a Honeywell HMC5883L magnetometer, an XBee Pro 868 radio module, a National Instruments sbRIO 9606 processing unit and a Hokuyo UTM30LXEW 2D time-of-flight laser scanner.
Christian Eling holds an MSc degree in geodesy and is a scientific assistant at the Institute of Geodesy and Geoinformation (IGG) of the University of Bonn.
Lasse Klingbeil received his Ph.D. in experimental physics in 2006. He heads the GNSS and mobile multi-sensor systems group in the IGG. Markus Wieland is a graduade mechanical engineer responsible for the mechanical and electrical design and for the control and readout of various sensor systems at the IGG.
Erik Heinz received his MSc in geodesy and geoinformation from the University of Bonn. He is a Ph.D. student at the IGG. Heiner Kuhlman is a full professor at the IGG. He has worked extensively in engineering surveying, measurement techniques and calibration of geodetic instruments.
Avyon, a sUAS (Small Unmanned Aircraft Systems) integrator and distributor, is using the Applanix APX UAV for its md4 fleet, to provide users with cost-effective direct georeferencing technology.
The integration of the Applanix APX-15 UAV on the md4-1000 and md4-3000 microdrones will offer solutions for unmanned aircraft while complying with weight and size restrictions for payloads. The APX-15 works seamlessly with all other airborne sensors such as digital cameras, LIDAR and other sensors, Avy0n said.
The APX-15 on the md4-1000 microdrone is on display at booth 1803 at the AUVSI Unmanned Systems 2015 show, being held May 4-7 in Atlanta, Ga.
“The integration of the APX-15 with md4-1000 and md4-3000 will provide users with a precision mapping capability, minimizing or eliminating the requirement for ground control points and making mapping missions more efficient,” said Mike Hogan, Avyon’s business development manager.
The APX-15 UAV on the md4 fleet will improve aerial mapping by eliminating GCPs (ground control points) for triangulation, as well as reduce the amount of overlap in the surveying process. This will increase efficiency and effectiveness for area flown per mission and the post-mission data processing, Avyon said.
“We recognize the need to provide the growing UAS mapping market with the same highly efficient solutions that we pioneered for airborne mapping over 15 years ago,” said Joe Hutton, director of Inertial Technology and Airborne Products at Applanix Corporation (xyHt pg. 14). “We are now offering a cost-effective solution that meets the size, weight, power and cost requirements of small UAS, and maintains the Applanix pedigree for quality and performance. We are pleased Avyon has partnered with us. The md4-1000 DMS-UAV is a powerful new solution.”
CompassData, a provider of geospatial data and services, announced that its CompassTA elevation accuracy software has received OCIO-ITS certification from the U.S. Department of Agriculture (USDA). Certification allows 40,000 USDA users the opportunity to utilize CompassTA software for elevation accuracy verification of LiDAR point clouds, digital elevation models (DEM), and other raster data sets.
“This certification provides assurance to our current and future USDA clients they are using a data verification tool that has been thoroughly scrutinized and tested by their own internal auditing process,” said Jeff Barker, CompassData product manager.
USDA certified the CompassTA software through the Office of the Chief Information Officer – Information Technology Services (OCIO-ITS) within the Device Deployment Services Branch.
Earlier this year, CompassData received DO-200A approval from the Federal Aviation Administration (FAA) to use its CompassAA software and ground control points (GCPs) to verify the accuracy of satellite and aerial imagery for the creation of certain aviation products.
CompassTA and CompassAA are software tools in CompassData’s CompassV&V line of Verification and Validation products. Based on the popular Topo Analyst and Accuracy Analyst software tools CompassData purchased from Spatial Information Solutions (SIS) in early 2014, the rebranded CompassV&V products include CompassAA, for orthorectified image verification, and CompassTA, for QA/QC of elevation data.
For 20 years, CompassData has performed custom GCP collection for clients in the geospatial profession and archived those points in a database for commercial sale to other end users. The CompassV&V tools are used extensively with custom and archived GCP to verify the accuracy of geospatial imagery, surface and elevation models and many other spatial products.
Used by numerous U.S. federal agencies under the SIS brand names, CompassV&V tools are content enhancement solutions that automate map accuracy verification and eliminate manual processing, ensuring consistent quality control of geospatial products backed up by standardized reporting procedures. Both tools establish automated workflows and generate standards-based documentation delivered along with end products.
“Since acquiring and rebranding the CompassV&V tools, we have made administrative upgrades to enhance the user experience,” said Barker. “Additional improvements are in the works.”
Leveraging the CompassV&V software tools, CompassData has expanded its custom Validation Service using GCPs. This service is offered for clients who prefer, or are required, to have an independent third-party perform quality assurance and supply verification reports, CompassData has licensed professionals on staff that perform Validation Services using high-quality GCPs along with the CompassV&V tools. The CompassData team can conduct this service faster and at lower cost than other firms that have to obtain their own GCPs.
