TOPODRONE, a Swiss-based designer and manufacturer of high-precision lidar equipment for installation on drones, vehicles and backpacks, launched AQUAMAPPER, a UAV-based solution for bathymetric surveying and marine construction.
AQUAMAPPER contributes to a complete set of photogrammetry, lidar and bathymetry surveying solutions from TOPODRONE. The product, compatible with the DJI Matrice 300 RTK, provides a combination of high-speed efficiency (up to 14 km/h) and accuracy mounted on a UAV. The application areas include an open sea bathymetric survey up to 100m depth, quantity survey and calculation of sediments and periodic maintenance survey of storage pools.
Photo: TOPODRONE
The new data-collecting device has been successfully used alongside TOPODRONE’s LiDAR ULTRA equipment for airborne surveying at a highway construction project in one of the toughest terrains in Romania, EU, including 7 tunnels, 24 bridges and 18 viaducts. The project was performed by the Romanian company GRAPHEIN TOPO SA to deliver a full digital twin of a studied area.
Photo: TOPODRONE
The TOPODRONE LiDAR ULTRA on board a DJI M300 drone was used to capture laser scanning data from an altitude of 100 m to 120 m over rugged terrain forest area to cover a corridor 32 km long and 400 meters wide in 14 flights while AQUAMAPPER connected to the same DJI M300 drone performed a bathymetric survey over six river crossings.
“The key advantage of the new bathymetric equipment from TOPODRONE is the ability to capture a riverbed with centimeter-level accuracy with high speed in fully automatic mode and without using any boat,” said Andrei Sueran of GRAPHIEN TOPO SA. “The combination of an echosounder, GNSS and inertial measurement system helps to get accurate results after data post-processing.”
Founded in 2012, GeoSLAM is a provider of mobile scanning solutions with proprietary high-productivity simultaneous localization and mapping (SLAM) software to create 3D models for use in digital twin applications.
The addition of GeoSLAM is expected to significantly expand and accelerate Faro’s market growth in the mobile scanning space.
“We are thrilled to add GeoSLAM’s handheld 3D scanning technology to our portfolio of cutting-edge data capture solutions,” said Michael Burger, Faro president and CEO. “Faro now offers the industry’s broadest set of 4D data-capture solutions, including 360° camera-based images, mobile scanning and stationary high-accuracy laser scanning, allowing customers to balance the need for accuracy, speed and detail depending on their requirements. These capture technologies provide the foundation for our 4D digital-reality-based SaaS (software-as-a-service) offering that will allow customers to access multiple 4D data sources for visualization and analysis through a single user experience. We welcome the GeoSLAM team to our Faro family.”
“Joining with Faro represents the next step in the growth of GeoSLAM and the establishment of mobile mapping as a driver for growth in the way businesses map and understand their spaces,” said Andy Parr, GeoSLAM CEO. “Both companies share a vision of the importance of mobile scanning in the burgeoning digital reality capture market.”
GeoSLAM reported £14.5 million in revenue with 18% EBITDA in the fiscal year ending March 31, 2022. EBITDA, a non-GAAP measure, is calculated as net income/loss before interest (income) expense, net, income tax expense (benefit), foreign exchange rate variance, and depreciation and amortization. The transaction closed on Sept. 1, funded with available cash reserves and equity consideration.
Under terms of the agreement, GeoSLAM shareholders received a cash payment of £22.0 million and 495,562 shares of Faro stock subject to customary lock-up provisions. Faro expects the acquisition to be accretive to Non-GAAP EPS in 2023.
Faro serves the markets of architecture, engineering and construction (AEC); facility operations and maintenance; 3D metrology; and public safety analytics.
U.S. CAD is working with Pointfuse to help shape the future of point cloud processing software. This collaboration began upon U.S. CAD completing Pointfuse’s authorized training program.
U.S. CAD is an architecture, engineering and construction technology consultant that has 20 years’ experience in combining BIM content creation and 3D modeling with application development and configuration services. U.S. CAD is a Pointfuse distributor and has been working with Pointfuse to align Pointfuse’s software with its existing product and service portfolio, learn the full capabilities and ROI of the software, and discover product development functionality.
Pointfuse software converts millions of individual measurements captured by laser scanning, photogrammetry and mobile mapping systems into useable 3D mesh models. Pointfuse is unique in automatically segmenting the mesh into discrete, selectable surfaces by identifying objects based on their geometry and assigning unique identifiers to them, the company said.
“We already knew the addition of Pointfuse point cloud processing software to our market offering would release the power of laser scanning, photogrammetry and mobile mapping systems for a range of digital construction, facilities management and virtual design workflows,” said Jeff Rachel, president at U.S. CAD. “However, working with Pointfuse to complete the training program we now truly understand how to help and support our customers as they integrate these technologies within their organizations.”
The new version of all-in-one MultiStation addresses a wide range of needs. (Photo: Leica Geosystems)
The new Nova MS60 MultiStation combines upgraded, faster 3D laser-scanning capabilities, GNSS connectivity and digital imaging with a high-end total station. According to Leica Geosystems, part of Hexagon, the multi-station brings sensor fusion to the “next level.”
The MS60 features several laser scanning updates, including an fast scanning speed of up to 30,000 points per second, optimized scan area definitions, adapted scan managements, and an improved scanning path for zenith scans.
It is also equipped with the unique AutoHeight feature, enabling users to save time by automatically measuring the instrument’s height with a simple button press. Measurement professionals can make decisions directly in the field, performing point-cloud analysis such as flatness analysis and as-built checks in the Inspect Surface app of the MS60.
“Scan data combined with traditionally measured points, whether it’s from the total stations or the GPS receivers, is one of those immediate deliverables that help our clients see what we’re doing. With the scan data of the Leica Nova MS60 MultiStation, we can graphically show — the same day it is collecting — the locations in the field to any person,” said Donald Smith, senior land surveyor and principal at BL Companies. “When you deliver on time and provide customers with a deliverable they can see, you’ve just got yourself a recurring client.”
The MS60 speeds up workflows by combining technologies in this all-in-one instrument. The MultiStation total station offers advanced imaging, scanning capabilities and GNSS connectivity. With Leica Captivate field software, all measurement and scanning data can be visualized in 3D for quality and completeness checks.
MS60 users can seamlessly transfer all data into Leica Infinity software to manage, process, analyze and perform a quality check. The MultiStation helps users deliver projects on time, save money and have high flexibility in the field.
“The MS60 merges data in a multi-level process — total station measurements are complemented by 3D point clouds, which are automatically registered and coloured by the image information. All data perfectly fits within the same coordinate system, globally referenced by GNSS measurements or by measuring known points,” said Falko Henning, senior product manager at Leica Geosystems.
“Unlike other measurement devices, the MS60 offers familiar total station capabilities and scanning functionality to fulfil job requirements on site.” Henning said. “The operator can use the red laser pointer to perform reflectorless measurements for direct remedial work on-site or stakeout points and use the field controller even while a scan is performed.”
Pointfuse has released the latest version of its advanced point cloud processing software that converts the millions of individual measurements captured by laser scanning and photogrammetry.
Featuring new streamlined classification to ensure maximum efficiency and multicore processing for unlimited conversion power, the new version of Pointfuse is set to transform workflows within digital construction, facilities management and virtual design applications.
“Pointfuse is designed to make the use of point cloud data more accessible by removing many of the traditional barriers to use,” said Mark Senior, regional sales director at Pointfuse. “Obstacles such as processing time and computer power, incompatibility within existing workflows and outputs files that are large and complex; these have all been obliterated with the latest Pointfuse release.”
Pointfuse now includes a new streamlined workflow which makes object classification easy, using templates and shortcuts to ensure maximum efficiency. This ability to classify objects within Pointfuse has had a huge impact on how as-built data is utilized within digital design workflows; being able to quickly compare specific as-built objects with the design enables more accurate clash detection, reducing the number of false clashes being flagged.
IFC (Industry Foundation Classes — an open format data model that is intended to describe architectural, building and construction industry data) templates can also be created and edited for specific applications. With applications including architectural, MEP and HVAC, selected objects can be classified and mapped to ensure compatibility with onward workflows.
Pointfuse also includes a new conversion engine which uses multicore processing to manage and enable unlimited point cloud conversion to provide real scalability. In addition, Pointfuse’s mesh models are intelligently optimized, reducing the working data size by a factor of up to 100, making them easy to share with online 3D collaboration platforms, such as BIM 360, 3D Repo, Revitzo and Trimble Connect.
“Using Pointfuse we can create intelligent 3D mesh models in a fraction of the time,” commented Ben Callan, BIM coordinator in global construction services company ISG’s UK Fit Out business. “This accelerated modelling and reduced risk of error contributes to a direct reduction in costs when compared against traditional methods of modelling and point cloud data analysis. The easy to use, easy to consume outputs are also paving the way for new applications of the data including existing versus design clash avoidance and checks of temporary works against required construction activities.”
3D Services is expanding its services to offer underwater 3D laser scanning in addition to standard 3D laser scanning and aerial surveying.
This new service is designed for underwater inspections. Previously, sonar technology was the main solution for underwater inspections, modeling and measurements. However, sonar does not provide the level of detail necessary for many underwater projects, the Connecticut-based company said.
3D Services is able to provide high-resolution scans and accurate 3D models of underwater structures or topography. It can instantaneously and repeatedly capture accurate data points within less than 1 mm and with a resolution of .01 mm.
The high detail provides the data needed to avoid risks and serious liability-related costs, as well as provide higher quality, more accurate data than before.
Integration for infrastructure monitoring, navigation
By Desislava Staykova and Nico Zill
Rapid development in the technology of combined sensors within complex systems has taken place over the last decade. Such systems provide different accuracy levels, offering the possibility of use in application areas such as surveying, railway and automotive engineering, land administration, and for navigation purposes.
Multi-sensor integration and fusion is a comprehensive process of reading and combining sensor signals to ensure a higher level of data reliability and accuracy. Input data from every sensor and further combination with specially developed algorithms ensures the complete identification of observed features, which would be impossible with data from each individual sensor operating separately.
Because of its flexibility and the possibility for fast and continuous data measurement, multi-sensor integration and fusion has evolved rapidly in different areas. The object of this article is to overview the use of high-end and low-cost system complexes and software solutions for the purposes of the engineering geodesy, transportation and navigation.
Deformation Monitoring
Geodetic measurements for monitoring and displacements analysis of various engineering objects have always played an important role in maintaining structures like bridges, dam walls, building columns, wind power generators, and other construction.
This requires properly designed network schemes enabling continuous and highly accurate measurements. For such angular and length measurements of millimeter-level accuracy that must be performed in intervals of minutes, hours or a day, standard total stations are being replaced by automated ones (ATS) comprising precise servomotors, automatic target recognition sensors, electronic inclinometers, self-calibration control systems and other sensors.
The synchronized process of high-accuracy measurements (angular accuracy better than one second and distance accuracy better than one millimeter) and simultaneously adjustment software enables real-time or post-processing deformation monitoring and analysis. This type of hardware and software combination is often used during the life cycle of a project for construction and reconstruction of objects and for regular monitoring of the object’s stability.
Terrestrial Laser Scanning. The need for precise modeling and geometrical characterization of large structures and open areas as dams, mines, landslides and others cannot be covered by traditional surveying methods which require the use a huge number of points for describing the object’s surface. The development of laser scanning technology in the last decade offers a new way for deformations measurements and becomes part of the infrastructure monitoring.
The high scanning speed, dense measurement of huge numbers of points and high accuracy gives terrestrial laser scanning (TLS) an advantage other technologies used for large structural monitoring. Compared with the technologies using single point monitoring approaches where the displacements detection is limiteded to specific benchmarks, TLS provides high data redundancy. Combined with proper software products, this technique offers the possibility for high-accuracy surface modeling and displacement detection at the millimeter level. The scanned object consists of a large number of points, which allows implementation of mathematical algorithms for modeling and analyzing the object’s behavior.
Another advantage of TLS as a remote sensing measurement tool is the minimized impact of the operator over the observed points and network.
A new method for structural monitoring has emerged recently, comprising the advantages of the TLS, GNSS, geotechnical and meteo-sensors, enabling wide-area coverage and surface monitoring. One such tool is shown in Figure 1.
Figure 1. Terrestrial laser scanning combined with GNSS and other sensors enables wide-area coverage and survace monitoring. (Images courtesy of Leica Geosystems)
Mobile Laser Scanning
For different navigation purposes, for monitoring and investigation of wide areas, static measuring methods are being replaced by complex mobile measuring combinations of both high-end and low-cost sensors, to ensure fast, continuous and accurate data acquisition.
Recently mobile laser scanning (MLS) has experienced rapid development and proved its usage particularly in the railway and automotive sectors, for deformation analysis, for monitoring and documentation of as-built street and railway networks and the corresponding infrastructure objects.
MLS for Rail and Road. The advantages of MLS for fast, high-accuracy and complete scanning of the surroundings make it an important part of current railway and road conditions monitoring.
Continuous data acquisition and processing minimizes operator errors, and significantly reduces the time for performance of the surveying work and a-posteriori data analysis.
Localization and recognition of infrastructure objects forming part of railway and road environment has long been of primary importance in the transportation sector.
For determination and documentation of as-built railway and street networks from acquired data, Technet-Rail (Berlin, Germany) developed two software solutions, SiRailScan and SiRoadScan, for point-cloud analysis. The integrated mathematical algorithms ensure high-accuracy extraction and adjustment of the as-built left rail, right rail and center line, as well as of the roads’ border lines.
The adjusted geometry forms the basis for driving speed control tests, determination of the as-built environment for clearance detection and documentation, investigation of catenary wire deviations, ballast and road settlements, traffic signal positions, and any changes in the existing situation (see Figures 2 and 3).
Figure 2. Adjusted as-built rail geometry with SiRailScan used as basis for performance of clearance analysis and documentation in chainage based railway system.Figure 3. Adjusted with SiRoadScan road border lines. Detection and recognition of the roads signals.
In response to the growing interest in application of the MLS technique and a-posteriori data adjustment for monitoring purposes, Technet-Rail developed additional tools for deformation analysis of structures such as tunnel bodies, railway bridges and road surfaces. The integrated software solutions enable comparison between the designed and as-built situation, epoch-wise analysis, modeling of the structure, development into 2D followed by color-coded deformation map (see Figures 4 and 5).
Figure 4. Tunnel deformation analysis performed with SiRailScan based on the as-built rail geometry. Automated calculation of differences between designed and as-built tunnel structure.Figure 5. Tunnel deformation analysis with SiRailScan based on a pre-defined form and direction.
MLS for Navigation. Multi-sensor integration is the basis for operation of the moving measuring systems integrating hardware devices such as laser scanning devices, GNSS, inertial measurement units (IMU), distance measuring instruments (DMI) and specific software algorithms for data synchronization. A milestone in the development of such systems is the measurement and navigation in indoor places or in areas with low or no GNSS coverage.
The need for safe and reliable navigation in transportation systems such as train control systems, intelligent vehicle systems, system tracking, in urban environments, underground areas, and other areas with no available GNSS signal stimulated much research in the area of multi-sensor integration and fusion. the main scope of some studies is the integration of different sensors delivering information for the attitude, velocity, acceleration such as the IMUs, inclinometers, wheel sensors, and correspondent filtering algorithms to achieve the best possible position accuracy without usage of GNSS signals.
Conclusion
For decades, infrastructure objects such as dam walls, bridges, tunnels, roads and railway tracks form a substantial part of civil engineering and engineering geodesy. The integrity of their structure requires deep knowledge of the behavior of these objects and the various methods for their optimal and high accurate monitoring. The rapid development evolution of multi sensor integration in combination with laser scanning technology makes it an essential method for accurate, continuous and dense measurement for the purposes of the engineering surveys.
Desislava Staykova and Nico Zill are engineers with Technet-rail 2010 GmbH, Berlin, Germany.
2G Robotics has delivered two deep-rated ULS-500 subsea laser systems to Oceaneering International Inc.’s business unit, Oceaneering Survey Services, a provider of deepwater seafloor mapping and subsea surveys.
Including these two ULS-500 systems, Oceaneering now uses six of the ULS-500 systems with its autonomous underwater vehicles (AUVs) as part of its advanced survey and inspection services for assessing pipeline and flowline integrity for the oil and gas industry.
The ULS-500 deep-rated system. (Photo: 2G Robotics)
Oceaneering has been using the ULS-500 since 2013 to perform dynamic flowline and pipeline inspections with its AUVs, and most recently used the 2G Robotics ULS-500 system to inspect 2,500 kilometers of pipeline and flowline.
The ULS-500 can be used to perform high-quality stationary scans, but the system delivers greater operational value when integrated with subsea vehicles to perform dynamic scanning, 2G Robotics said. The ULS-500 is designed for dynamic scanning with development focused on subsea vehicle integration, high sample rates, and timing synchronization for efficient and accurate data acquisition.
The ULS-500 uses PPS (pulse per second) time synchronization because it provides better timing accuracy than a standard NTP (network time protocol) time synchronization approach, ensuring better data accuracy.
The 3D point cloud models generated by the ULS-500 provide Oceaneering with the detail needed to accurately assess pipelines and flowlines, and measure displacements and deformations.
(Center) The National Oceanic and Atmospheric Administration (NOAA) used the ULS-500 to explore the Monohansett shipwreck site at the Thunder Bay National Marine Sanctuary. The ULS 500 captured detailed 3D models of the Monohansett at a range of approximately 3m to 5m. (Credit: 2G Robotics)
Four point clouds, nonregistered, of georeferenced images from four UAV flights.
By Christian Eling, Lasse Klingbeil, Markus Wieland, Erik Heinz and Heiner Kuhlmann
Direct georeferencing with onboard sensors is less time-consuming for data processing than indirect georeferencing using ground control points, and can supply real-time navigation capability to a UAV. This is very useful for surveying, precision farming or infrastructure inspection. An onboard system for position and attitude determination of lightweight UAVs weighs 240 grams and produces position accuracies better than 5 centimeters and attitude accuracies better than 1 degree.
Data acquisition from mobile platforms has become established in many applications recently, particularly using unmanned aerial systems (UASs). Unlike other mobile platforms, unmanned aerial vehicles (UAVs) can overfly inaccessible and also dangerous areas. Furthermore, they can get very close to objects to collect high-resolution data with low-resolution sensors, and they enable approach from all viewing directions without physical contact. UAVs now see use in precision farming for phenotyping or plant monitoring, and in infrastructure inspection and surveying.
Data acquisition from mobile platforms has become established in many applications recently, particularly using unmanned aerial systems (UASs). Unlike other mobile platforms, unmanned aerial vehicles (UAVs) can overfly inaccessible and also dangerous areas. Furthermore, they can get very close to objects to collect high-resolution data with low-resolution sensors, and they enable approach from all viewing directions without physical contact. UAVs now see use in precision farming for phenotyping or plant monitoring, and in infrastructure inspection and surveying.
This article addresses lightweight UAV use for mobile mapping and uses the term micro aerial vehicle (MAV) throughout. MAVs can generally be characterized as having a weight limit of 5 kilograms and a size limit of 1.5 meters.
We focus on the development of a real-time capable, direct georeferencing system for MAVs, since spatial and time restrictions often exclude the possibility of deploying ground control points for an indirect georeferencing. The demand for the real-time capability results from the aim to also use the georeferencing for autonomous navigation of the MAV and to enable a precise time synchronization of the onboard sensors. Furthermore, a real-time direct georeferencing also offers the opportunity to process collected mapping data during flight.
Mapping on demand. The goal of this research project, funded by the Deutsche Forschungsgemeinschaft (DFG), is to develop an MAV that can identify and measure inaccessible three-dimensional objects by use of visual information. A major challenge within this project comes with the term “on demand.” This means that apart from the classical mapping part, where 3D information is extracted from aerial images, the MAV is intended to fly fully autonomously on the basis of a high-level user inquiry. During the flight, obstacles must be detected and avoided. To extract semantic information that can be used to refine the trajectory planning, the mapping data has to be processed in real time. When the georeferencing information is used as initial values for the bundle adjustment, the image processing can be significantly accelerated.
Figure 1 shows the current MAV platform developed in this project. We customized an MAV kit to a coaxial rotor configuration, replaced the centerplates with more stable carbon-fibre plates to stabilize the system, and installed the direct georeferencing and the mapping sensors. The two stereo camera pairs, pointing forward and backward, act as an additional sensory input for the position and attitude determination; the 5M-pixel industrial camera with global shutter is the actual mapping sensor. The PC board is used for onboard image processing, flight planning and machine control; the Wi-Fi module enables a connection to a ground station.
Figure 1. The MAV with mapping and georeferencing sensors, developed for the research project Mapping on Demand.
Although the direct georeferencing system must be small and lightweight, accuracy requirements for its position and attitude determination are high. Generally, these accuracy requirements are different for the machine control, navigation and mapping purposes.
In our project, the MAV is intended to maintain a safety distance of about 0.5 meter to obstacles. Hence, a position accuracy of 0.1 meter is sufficient for the navigation. The absolute attitude accuracy should be in the range of 1 to 5 degrees. For machine control, relative information is more important, and for this the accuracies should be slightly higher.
For mapping purposes, the positions and attitudes have to be known better, since the absolute georeference of the final product (for example, a high-resolution 3D model of a building) is based on the positions and attitudes from the direct georeferencing system. Therefore, the position accuracy should be in the range of 1–3 cm and the attitude accuracy should be better than 1 degree. The relative accuracy of the exterior camera orientation can be improved by a photogrammetric bundle adjustment, but systematic georeferencing errors should be avoided.
To summarize:
The weight of the system has to be less than 500 grams (g), to be applicable on MAVs.
Especially for the control and navigation, the system has to be real-time capable.
All sensors have to be synchronized and outages of single sensors should be bridgeable by other sensors.
The system is intended to provide accurate positions (σpos < 5 cm) and attitudes (σatt < 1 deg) during flights.
The integration of data from additional sensors, such as cameras, should be possible.
The ability to include additional sensors to the system was, apart from the size and the weight constraint, the main reason for developing a proprietary system instead of using a commercial unit with similar capabilities.
Direct Georefencing
The current version of the system weighs 240 g without GPS antennas (see figure 2). To reduce weight, the antennas were dismantled, reducing their weight from 350 g to 100 g. However, since the antenna reference point got lost in this process, the antennas had to be recalibrated in an anechoic chamber for further use. By comparison to the original antennas, the dismantling led to significant changes in the phase center offsets (circa 4 cm in the Up, < 1 mm in the North and East component) and in the phase center variations (< 5 mm) of the antennas.
Figure 2. The direct georeferencing system.
Figure 3 shows a flow chart of the direct georeferencing system with the sensors and the main calculation steps. The system consists of a dual-frequency GPS receiver, a single-frequency GPS receiver, an inertial measurement unit (IMU) and a magnetometer. The dual-frequency receiver is the main positioning device. Together with the GPS raw data from the master station (carrier phases ϕM, pseudoranges PM), which is transmitted via a radio module, the data of the dual-frequency receiver (ϕR, PR) is used for an RTK positioning, leading to centimeter position accuracies.
Figure 3. Flowchart of the direct georeferencing system.
In collaboration with the data of the single-frequency receiver (ϕB, PB), the data of the dual-frequency receiver is also used for GPS attitude determination. The corresponding GPS antennas of these two receivers form a short baseline (baseline length = 92 cm) on the MAV. The determination of the baseline vector in an e-frame (Earth-fixed) enables yaw and the pitch-angle determination.
The tactical-grade micro-electro-mechanical (MEMS) IMU, which includes three-axes gyroscopes, accelerometers and magnetometers, provides angular rates (ω), accelerations (a) and magnetic field observations (h) with high rates (100 Hz) for position and attitude determination. To be unaffected by the electric currents as much as possible, an additional magnetometer is placed on the outer end of one of the rotor-free MAV arms.
The direct georeferencing system further consists of a processing unit, which is a reconfigurable IO board, including a field programmable gate array (FPGA) and a 400-MHz processor. In this combination, the FPGA is used for fast parallel communication with the sensors. Afterwards, the preprocessed sensor data are provided to the 400-MHz processor via direct memory accesses, avoiding delays and supporting the system’s real-time capabilities. Finally, the actual position and attitude determination is carried out on the 400-MHz processor.
Methodologies
All position and attitude determination algorithms running on the system were developed in-house. Generally, the integration of these steps could be realized in one tightly coupled approach. Nevertheless, in the current implementation, we decided to separate the different raw data calculation steps, and we only use interactions at the level of parameters. This approach has the advantage that the integration is more reliable and more practical in the real-time programming.
GPS/IMU integration. In this calculation step, all available sensory input is fused to determine the best position and attitude of the system that is currently available. The GPS and the IMU measurements complement each other well, since the IMU provides short-term stable high-rate (100 Hz) data, and the GPS provides long-term stable low-rate (10 Hz) data.
The GPS/IMU integration can be separated into the strapdown algorithm (SDA) and the Kalman filter update. In the SDA, the high-dynamic movement of the system is determined integrating the angular rates and the accelerations of the MEMS IMU in real time. Because the SDA drifts over time, the long-term stable measurements of the magnetometer and the GPS receivers are needed to correct and bound the drift of the inertial sensor integration, which is realized in an error state Kalman filter.
In the GPS/IMU integration algorithms, the navigation equations of the body frame (b-frame) are expressed in an e-frame. Therefore, the full state vector x includes the position xep and the velocity vep, represented in the e-frame. For the attitude representation a quaternion q is used. Finally, the accelerometer bias bba and the gyro bias bbω are also estimated:
The observations in the measurement model are:
the RTK GPS position xea of the dual-frequency RTK GPS antenna reference point, expressed in the e-frame,
the GPS attitude baseline vector Δxeb, expressed in the e-frame,
the magnetic field vector hb, expressed in the b-frame.
Because the reference point of the RTK GPS antenna is not identical to the system reference point, a lever arm between the system and the antenna reference point must be regarded in the measurement model of the RTK GPS positions. From calibration measurements, the coordinates of the lever arm are precisely known in the b-frame.
In the SDA, a coupling between the accelerations, measured by the IMU, and the positions, measured by the RTK GPS, exists. Due to this coupling the yaw angle can be observed, but only in the presence of horizontal accelerations.
To determine an accurate and reliable yaw angle for every motion behavior, the short GPS baseline is realized on the MAV. A significant challenge in processing this baseline is the ambiguity resolution, because only single-frequency GPS observations can be used. Empirical tests have shown that the ambiguity resolution of a single-frequency GPS baseline generally takes several minutes. Among other strategies, we use the additional information from a magnetometer to improve the ambiguity resolution and to actually enable an instantaneous ambiguity fixing during kinematic applications.
Ferromagnetic material on the UAV and high electric currents of the rotors create significant disturbances of the magnetometer during flight. While the influence of the material can be compensated by calibration procedures, the influence of the dynamically changing electric currents are more challenging. To minimize them, the magnetometer is placed at the outer end of a rotor-free arm of the MAV. Also, the measurement model is arranged so that magnetic field observations only have an impact on the yaw determination in our algorithms.
RTK GPS Positioning. RTK GPS positions are calculated in real time with a rate of 10 Hz. These RTK algorithms are in-house developed, although commercial and open-source solutions are available. The main reasons for developing custom software are the following:
Integration of other sensors and/or solutions is possible, to improve ambiguity resolution and cycle-slip detection.
In commercial software, there is generally no access to the source code.
In the development of a real-time capable system, the software must meet the requirements of the operating system running on the real-time processing unit.
Generally, the RTK GPS algorithm complies with a single baseline determination (one master, one rover), where the master station remains ground-stationary and the rover is onboard the MAV.
To resolve the ambiguities and finally to determine the RTK GPS positions, the parameter estimation is performed in three steps: float solution, integer ambiguity estimation and fixed solution.
The float solution is realized in an extended Kalman filter (EKF). Beside the rover position, represented in the e-frame, the EKF state vector xSD also contains single-difference (SD) ambiguities N j on the GPS L1 and the GPS L2 frequencies. The reason for estimating SD instead of double-difference (DD) ambiguities is to avoid the hand-over problem that would arise for DD ambiguities, when the reference satellite changes.
To allow for an instantaneous ambiguity resolution, the observation vector l consists of DD carrier phases Φjkrm and DD pseudoranges Pjkrm on the GPS L1 and the GPS L2 frequencies.
In the current implementation, a random walk model is assumed as a dynamic model of the MAV in the EKF. Even if this is a simple model, it complies with the movement of the vehicle, when the process noise is chosen appropriately.
The float solution procedure provides real-valued ambiguities and their covariance matrix. These ambiguities now must be fixed to correct integer values, to fully exploit the high accuracy of the carrier phase observables. We applied the MLAMBDA method for integer ambiguity estimation.
Finally, a decision must be made whether or not the result of the integer ambiguity estimation can be accepted. This is done by the simple ratio test. With the ambiguities fixed, the final rover position xae is estimated with cm accuracies.
Usually, the time to fix the ambiguities with the algorithm takes a few epochs, but often the ambiguities can be fixed instantaneously. Once ambiguity resolution has been successful, the ambiguities can be held fixed, as long as no cycle slip or loss of lock of GPS signals occur.
Due to the GPS/IMU integration, we have a precise prediction of the RTK GPS positions between two epochs. Thus, the integration of the inertial sensor readings enables us to detect and also repair cycle slips very reliably.
The observations of the master receiver must be transmitted via radio to the direct georeferencing system. In practice, this data transmission can only be realized with a rate of 1 Hz. To be less dependent on this potentially unreliable master data transmission and the lower sampling rate, simulated master observations are used for RTK GPS position determination. Hence, in the actual processing, the true master observations are only used to update the simulation errors in the master task (figure 4), which have to be applied to correct the simulation results in the rover task.
Figure 4. Task scheduling of the RTK GPS algorithms.
GPS attitude determination. The GPS baseline is determined at 1 Hz. In contrast to the RTK GPS positioning, both antennas of the attitude baseline are mounted on the MAV, so that the complete baseline is moving. Furthermore, the baseline length is constant and known from calibration measurements. The GPS attitude determination also consists of the three steps: float solution, integer ambiguity estimation and fixed solution.
The float solution is also based on an EKF where the single-frequency SD ambiguities N j of the attitude baseline are estimated. Further parameters in the state vector are the baseline parameters and the first deviation of the baseline parameters.
As observations DD carrier phases ΦjkAB and DD pseudoranges PjkAB on the GPS L1 frequency are used. To improve the ambiguity resolution, the attitude from the GPS/IMU integration is added to the observation vector, by transforming the known b-frame baseline parameters into the e-frame. Finally, also the known baseline length can be added as a constraint to the observation vector.
In the integer ambiguity estimation, we apply the MLAMBDA method again. Due to the prior information about the attitude of the baseline, the float ambiguities can already be estimated with high accuracies in the float solution. If the ambiguities could not be fixed with the MLAMBDA method, we consider the 10 best solutions for further processing. Unreliable ambiguity parameters are eliminated in a random order, and the MLAMBDA method is applied again. Afterwards we use the ambiguity function method and the known baseline length to exclude false candidates of the 10 best solutions.
If only one solution remains, the ambiguities can be fixed to integer values. Tests have shown that this approach leads to an instantaneous ambiguity resolution success rate of about 95 percent.
Similar to the RTK GPS positioning, the IMU readings are also used to detect cycle slips for the attitude baseline determination, when the ambiguities have been fixed successfully. With ambiguities fixed, the baseline parameters can be determined with millimeter to centimeter accuracies. This leads to yaw angle accuracies in the range of 0.2–0.5 degrees, when the attitude baseline has a length of 92 cm.
Applications and Results
As mentioned, one goal of Mapping on Demand is 3D reconstruction from visual information. The opening image shows such results. During four flights. images were collected with a sampling rate of 1 Hz, and the position and the attitude of the camera was determined in real time using the direct georeferencing system. A bundle adjustment was processed using these positions and attitudes as initial values. Afterwards, dense point clouds could be generated from the oriented images using an open-source software package (PMVS). Due to georeferencing of the collected images, the point clouds are also georeferenced. The image shows results of four flights in one scene, to demonstrate consistency of the georeferencing.
Agriculture. In figure 5, georeferenced images were taken during a flight over a wheat field. The same process was repeated after two weeks. The difference of the respective point clouds, which were determined using the software Photoscan by the company Agisoft, reveals the plant growth at an interval of two weeks. These results show that the determination of plant growth rates, which usually result from time-consuming field work, can be done easily and with high resolution using MAVs. With the use of a direct georeferencing system, this process becomes even more efficient because the deployment of ground control points can be omitted.
Figure 5. Orthophoto of a wheat field (left) and the difference of the vegetation height, determined from the results of two MAV flights at an interval of two weeks (right).
Portable laser scanning system. The small and lightweight design of the direct georeferencing system offers several other opportunities for various applications. One example is the use of the direct georeferencing system in combination with a small, lightweight and low-cost laser scanner.
Terrestrial laser scanning has become an established technology for 3D data acquisition in surveying and mapping because laser scanners provide high-resolution data with high accuracies at high speed. However, for measurement of a complex scene, the laser scanner generally has to be moved to different viewpoints, and all measured scenes have to be registered and georeferenced, a significant increased effort. In contrast, with a directly georeferenced kinematic laser scanning system, complex scenes can be measured with little effort.
Figure 6 shows a portable laser scanning system we developed for kinematic laser scanning. It combines the direct georeferencing system with a low-cost, lightweight 2D time-of-flight laser scanner. Time synchronization and the point cloud calculation are directly realized on this unit.
Figure 6. A directly georeferenced portable laser scanning system for kinematic 3D mapping.
Figure 7 shows differences between a directly georeferenced point cloud, measured by the portable laser scanning system, and a terrestrial laser scanning point cloud, which was indirectly georeferenced using ground control points. Although there are some systematic errors visible, the differences are mostly less than 7.5 cm. The larger differences in the foreground (red) are a result of growing vegetation in the period between both scans. The systematic errors result from the system calibration between the laser scanner and the direct georeferencing system. We are working to improve these calibration methods.
Figure 7. Difference between the results of the directly georeferenced portable laser scanning system and the results of a terrestrial laser scan, which act as reference solution here.
Manufacturers
The MAV is based on a MikroKopter OktoXL assembly kit of HiSystems GmbH. It uses NavXperience 3G+C GPS antennas. The system consists of a dual-frequency NovAtel OEM 615 GPS receiver, a single-frequency u-blox LEA6T receiver, an Analog Devices ADIS 16488 IMU, a Honeywell HMC5883L magnetometer, an XBee Pro 868 radio module, a National Instruments sbRIO 9606 processing unit and a Hokuyo UTM30LXEW 2D time-of-flight laser scanner.
Christian Eling holds an MSc degree in geodesy and is a scientific assistant at the Institute of Geodesy and Geoinformation (IGG) of the University of Bonn.
Lasse Klingbeil received his Ph.D. in experimental physics in 2006. He heads the GNSS and mobile multi-sensor systems group in the IGG. Markus Wieland is a graduade mechanical engineer responsible for the mechanical and electrical design and for the control and readout of various sensor systems at the IGG.
Erik Heinz received his MSc in geodesy and geoinformation from the University of Bonn. He is a Ph.D. student at the IGG. Heiner Kuhlman is a full professor at the IGG. He has worked extensively in engineering surveying, measurement techniques and calibration of geodetic instruments.
