$50K prize purse for solutions to advance computer vision for time-sensitive mapping
The National Geospatial-Intelligence Agency (NGA), Johns Hopkins University Applied Physics Laboratory and NASA have launched a crowdsourcing competition, the “Overhead Geopose Challenge,” which is being implemented by DrivenData and HeroX.
The challenge seeks solutions that make overhead images more useful for time-sensitive applications like security and disaster response. Participants are competing for a prize purse of $50,000. To accept the challenge, visit this website.
The challenge seeks to build computer-vision algorithms that can effectively model the height and pose of ground objects for monocular satellite images taken from oblique angles.
Overhead satellite imagery provides critical time-sensitive information for disaster response, navigation, and security. Most current methods for using aerial images assume that the images are taken from directly overhead.
However, the first images available are often oblique. These camera orientations complicate useful tasks like image alignment and change detection that are needed to ensure that maps are accurate in dynamic environments.
Solvers will transform RGB images taken from a satellite to accurately determine each object’s real-world 3D structure or “geocentric pose.” Geocentric pose is an object’s height above the ground and its orientation with respect to gravity. Calculating geocentric pose helps with detecting and classifying objects and determining accurate object boundaries.
NGA is continuously seeking novel concepts for assured positioning, navigation, and timing, said NGA Research Chief Scientist John Main. “We think the broader innovation community can help us support disaster response operations more effectively by exploring novel ways to understand where things are happening — or even better where things will be happening — during an emergency event. This is NGA Research’s goal and the reason we tap scientific and technical talent and ideas from national labs, universities, and businesses.”
“This is a hard computer vision problem that will benefit from the diverse approaches that a machine learning challenge can bring to bear,” said Greg Lipstein, co-founder and principal of DrivenData. “This is a great opportunity to bring together a fascinating dataset with a passionate community of experts to generate innovative solutions for disaster response and other time-sensitive needs”
“Our global network of problem-solvers is perfectly suited to support the advancement of state-of-the-art methods for using and understanding satellite imagery,” said Christian Cotichini, CEO, HeroX. “Having great minds come together for an initiative like this will help us more quickly and more effectively support people in times of need.”
Eligibility to Compete and Win Prizes
The challenge is open to anyone aged 18 or older not affiliated with DrivenData or the challenge sponsor and permitted to compete under the laws of the U.S. and their local jurisdictions.
Finalists will be determined by performance on a private test set of historical measurements, and bonus prizes will also be awarded for qualifying scores that reflect innovative approaches. All prize-winning approaches will be shared under an open-source license for learning and use by the community.
How are oblique views derived from aerial imagery?
Typically, a camera takes a field of view of 120 degrees (+/– 60 degrees either side of centerline). The nadir is straight down +/– 5 degrees either side, but everything beyond is considered oblique imagery.
Overlapping imagery is required to ensure clean images and to reduce the angle of obliquity. Too much of an oblique angle causes parallax, which distorts the image, so it is usual for imagery to overlap by 70% each pass, meaning that 30% either side of center is used, but everything except for a small path considered nadir is double imaged.
However, in the case of stereographic imagery, which is required for building a 3D mesh, the overlap has to cover the centerline of the last flight path, so the flights must be much closer together.
Oblique imagery allows 3D meshes to be created, which is a huge benefit to geospatial analysis. It allows the actual terrain to be measured not in a straight line, but in an actual topographic line that includes elevation changes for point-to-point distance.
Additionally, straight lines work when everything looks flat, but in reality straight lines are rare, and point-to-point measurements often have to take advantage of the existing terrain, avoiding steep terrain and aiming to stay on the highest ground to avoid marshy areas.
Oblique imagery also allows for mensuration, which is the measurement of the vertical based on the trigonometry of the sensor’s position and height compared to the target’s angle. More than one oblique image of the same target area allows for stereographic imagery for building the 3D meshes and seeing in 3D. Without the magic of oblique imagery, GIS would be a 2D science.
Nearmap has announced a national survey program providing true, high-resolution oblique imagery and derivative 3-D products.
Nearmap provides cloud-based subscription access to up-to-date 2-D orthomosaic aerial imagery. Using its patented HyperCamera2 technology, Nearmap is applying the same access model to the oblique aerial imagery market.
Because this new camera system provides a high degree of overlap from different angles, Nearmap can reconstruct the real world in stunning detail, producing not only high-resolution orthomosaic and oblique imagery, but also surface and terrain models, natural-color point clouds and textured 3-D meshes.
“This level of detail and scale of coverage of oblique imagery has never been available as a ready-to-use service for commercial and government needs until now,” said Patrick Quigley, senior vice president and general manager, U.S. of Nearmap. “The HyperCamera2 process maps reality, by capturing the tops, sides and view angles of locations, buildings and objects, providing specific details of what’s exactly on the ground.
Screen capture from a Nearmap 3D fly-through of Austin, Texas, rendered from Nearmap oblique Imagery.
Users will be able to immerse themselves in 3-D textured mesh models, improving analysis and design activities. They can see different elevations and line of sight using the 3-D information.
These features become important in many use cases, including airport or utility planning, or to determine the best location for a crane before a construction project. Other applications include wireless telecommunications network modeling, solar panel design, tactical resource deployment, real estate development promotion, property valuation, insurance underwriting and smart cities.
“3-D brings a whole new aspect of mapping reality to both commercial and government organizations,” said Rob Newman, CEO and managing director of Nearmap. “This new service will help industries plan, design, estimate, communicate and execute their plans — everything from major construction projects to solar installations on homes and businesses.”
Beginning in April, Nearmap has already surveyed oblique images in Las Vegas; Indianapolis; Austin, Texas; Omaha, Nebraska; Phoenix; Seattle; Denver; Kansas City, Kansas; Chicago and New York, and continues to add new areas.
By the end of 2017, Nearmap plans to complete surveying the largest urban areas covering more than half of the U.S. population — about 150 million people.
Nearmap imagery will be refreshed up to three times per year in these coverage areas — with three orthomosaic captures incorporating one oblique capture. Nearmap’s orthomosaic imagery already covers nearly 70 percent of the U.S. population dating back to 2014. “This gives our customers the aerial imagery services they need for their businesses and projects,” Quigley said.
Nearmap’s oblique imagery can be accessed in the MapBrowser interface or integrated into a customer’s own web application using Nearmap’s industry-standard API. Digital surface modeling is also available for export into GIS / CAD tools, including Esri’s ArcGIS Pro. Nearmap will soon enable similar access to the 3-D products.
Fugro is enhancing its mapping services with the introduction of an integrated software package that creates realistic, yet spatially accurate, high-resolution 3D building models using oblique imagery.
The product will benefit emergency responders during critical infrastructure planning, as well as those involved in infrastructure development, utilities and property management.
Fugro, partnering with Skyline Software Systems, provides infrastructure management professionals with robust new capabilities in oblique mapping, including detailed 3D models and view shed studies for advanced visualization and analysis.
Announcing a strategic partnership with Skyline Software Systems Inc., Mike Wernau, Fugro’s Oblique program manager explained, “We are now able to take a 2D oblique product and deliver a realistic 3D environment with enhanced viewing, query, analysis and reporting options. The value that users are going to discover as a result of this software fusion is something the market has really never seen before.”
The new partnership offers clients an integrated oblique viewing and 3D modeling software. The integration of TerraExplorer and PX Mapper transforms the application of 3D environments by using 2D oblique imagery to create high-resolution building models that are both realistic and spatially accurate.
Fugro’s oblique mapping solution includes high-resolution 360-degree oblique imagery and the PX Mapper visualization and analysis software. The Fugro/Skyline alliance allows oblique customers to experience Skyline’s automated modeling technology along with TerraExplorer’s optimized analytics capabilities including terrain analysis for flooding, contours, slope and volume metrics, lines of sight and view shed queries and shadow analysis.
“By combining oblique imagery with our high-quality 3D modeling tool and integrating those models with the unlimited capabilities of TerraExplorer Pro, we’ve created the optimal environment for GIS infrastructure,” said Eatay Ben Shechter, director of production at Skyline. “This versatile product supports countless real-world applications, where time-critical decision making is required, from urban planning to real-estate management, and multiple different emergency response scenarios.”
A flagship project employing this integrated 3D environment is already underway with completion in Summer 2017.
When I entered the civilian part of my GIS career as the GIS manager for the Atlanta Regional Commission, I tried to get first responders interested in GIS. Of course, in the early ’90s we were happy to be able to accurately draw points, lines and polygons on a piece of paper. Soon we had the luxury of ortho imagery as a backdrop for our GIS data, but I still couldn’t build a lot of enthusiasm among those first responders.
That changed completely when we started using metric oblique imagery provided by Pictometry. I realized that since we live in an oblique/3D world many non-GIS users had real difficulty visualizing objects or locations using two-dimension visualizations such as drawings, blueprints, maps or even ortho imagery.
By contrast, oblique views made visualization much easier for the vast majority of non-GIS users, and use of oblique imagery coupled with GIS tools exploded. Since then, many of us have been searching for faster, easier and cheaper ways to collect oblique imagery and video, and build 3D models.
For more than a decade, major defense contractors developed leading-edge systems to capture and exploit aerial imagery and video. Although effective, as one would expect of new custom technology, the systems were very expensive and out of reach for most local government agencies. Remote GeoSystems seems to have developed a system that leverages current technology to provide capabilities that may address some of those needs at a reasonable price.
Remote GeoSystems is in the business of capturing, displaying and managing “georeferenced” video and imagery. The company has designed and built high-end geospatial video recording systems for full motion video (FMV) and GIS mapping software primarily aimed at regulatory compliance of energy corridors, grids and critical infrastructure inspection applications.
Fortunately, my UAV is a DJI Inspire 1. I chose the Inspire because of its reputation, and because it seems to be the best combination of features needed for first-responder work at a prosumer price (about $3,500). The Inspire can record up to 4K video/12-mp stills, has a 94-degree field of view so there is no wide angle “fish-eye” distortion typical of an action camera, and has “Lightbridge” technology that permits positive control up to 3 miles and the ability to stream live 720p video (now 1080p) back to the ground controller.
The controller can feed large-screen video for command center group viewing via an HDMI output. Most important, the Inspire records GPS position data and altitude along with the video/imagery stream. (The DJI Phantom 3 Pro is a cheaper alternative that also records telemetry data, but if one upgrades to a 4K camera and the Lightbridge transmitter/receiver, the price approaches the integrated Inspire 1 price.)
An .srt file.
Since I’m always leery of marketing pieces and company demos, I wanted to try the system myself, and Remote Geo was happy to oblige. My first hands-on test was very satisfying. The LineVision software downloaded, unpacked and loaded quickly with no problems. I then recorded some aerial video of our condo building on Lake Guntersville near Huntsville, Alabama. I chose this building because it was convenient, safe to fly and a multi-story building in the open.
In addition to recording the video, one needs to turn on the DJI Inspire metadata recording to generate the .srt file. This is done in the DJI application “General Settings/Camera” by toggling “Video Caption” on. The .srt file was initially designed to provide altitude and location data as on-screen captions, but the data can be used as needed for other purposes.
When done with the flight and recording, transfer the video file and .srt file to your computer. Make sure the video file .mov/.mp4 and .srt file are in the same folder. Open LineVision and you will see an ArcGIS window. From the pull-down menu, load the video and you will instantly see the video play in a separate window with red position dots on the ArcMap view. As the video plays, the dot associated with the location of the UAV will turn yellow. If you click on any dot, the video will jump to that location/position on the video.
Here are screen captures of LineVision showing the ArcGIS view of an ortho image with red dots illustrating the path of the UAV:
One advantage of LineVision for first responders is that it is a complete package with ArcGIS embedded, all for a price well below $1,500. There is no need for a separate ArcMap license. Additionally, although LineVision Esri ArcGIS can display GIS data from online sources, if you have GIS data for your location loaded on your computer the system will operate in a disconnected remote environment. These sample screengrabs don’t do the system and video justice, since I recorded at 1080p rather than 4K. My laptop, this website and the reader’s playback equipment limit accurate playback of 4K content, so I did my work at 1080p.
I can envision a disaster-response scenario where the response team arrives on site, launches a UAV, and starts recording the scene. The captured video could then be loaded, viewed, indexed and cataloged with GIS data overlays on a laptop all in a matter of minutes, even in a disconnected environment. Hours, days or months later, finding the right video clip for analysis or forensics should be significantly easier and faster.
With the explosion of UAV hardware and software, it’s going to be an exciting year as new smaller, cheaper and more capable systems hit the market. Remote GeoSystems is working with UAV manufacturers to make LineVision capability available for many of the newcomers.
Leveraging UAV and LineVision capability, Skyline has worked with Remote GeoSystems to bring yet another capability: rapid 3D model creation. Taking appropriate geo-located frames of the video, Skyline uses its PhotoMesh software to build fully metric 3D models in short order. The full capability of this system and its 3D viewer TerraExplorer is so extensive that I will cover it in a future column, after this month’s ESRI Federal Users’ Conference. If you see me at the UC Feb. 24-25, please stop me and say hello.
DAT/EM Systems International has released the 7.1 edition of DAT/EM software products, including Summit Evolution, Landscape, Capture, MapEditor, Ortho+Mosaic, Airfield3D and Contour Creator. The advancements in the 7.1 DAT/EM Photogrammetric Suite represent the latest evolution in technology and are based on customer input and growth within the geospatial industry, the company said.
DAT/EM Photogrammetric Suite Version 7.1 Highlights
A new Multiple Project Solution combines different project types that have any input coordinate systems and displays them in multiple viewports. The viewport cursors move together where the diverse projects share the same geographic area. This allows use of imagery from sources such as aerial, orthophoto, ADS, and satellite RPC at the same time.
Oblique imagery support in the aerial project type. Oblique options will apply for any imagery where omega and phi are greater than 15 degrees.
Enhanced printing options, such as print preview, print in anaglyph, print with boundaries, print with full superimposition, virtually print to TIFF, and print to a higher resolution.
A Direct Linear Transform (DLT) model has been added to Summit close range projects. This model allows Summit to orient photos without a known camera calibration.
Named Image Adjustments is a new tool that saves image adjustment profiles for repeated use. It can also link to a Channel Map configuration.
Aerial projects can now define image scale, earth curvature correction, and refraction correction by a new Use heights method, which defines individual model scales and is used in steep terrain.
A second generation beta of the Global Mapper Extension. The Extension connects Blue Marble Geographics Global Mapper with any Summit Evolution edition (Professional, Feature Collection or Lite).
LandScape offers a new slider to adjust the intensity of color.
All DAT/EM products support new file formats, including LAZ format for both read and write operations, all LAS 1.4 for read only, and LAS 1.4 non-waveform formats for read and write.
Icaros Inc., a provider of advanced aerial remote sensing and 3D visualization solutions, will demonstrate the Icaros Measurement Tool (IMT) at the 2014 InterGeo Conference in Germany. Designed for use with oblique aerial imagery, IMT is a simple but powerful photogrammetric visualization application based on ArcGIS technologies, the company said.
To view a demonstration of the IMT, visit Icaros in stand #A4.009 Hall 4.1 at the InterGeo Conference and Trade Fair being held in Berlin on October 7-9.
The IMT is a universal oblique measurement tool. It enables customers working within Esri’s GIS environment to view and measure structures in oblique aerial imagery captured by any commercial oblique sensor system, including those from Pictometry, Vexcel/Microsoft, IGI, Leica, and Midas. IMT works equally well measuring images captured by oblique sensors mounted on unmanned aerial vehicles (UAV).
“Icaros is opening the oblique market to all GIS users by enabling them to purchase imagery from any of the growing number of aerial oblique data providers,” said Richard Baumgartner, vice president of Business Development at Icaros. “For too long, the purchase and use of oblique imagery have been limited by proprietary collection systems and metadata formats.”
Access to additional oblique vendors will reduce data costs in the long run, Icaros said. In addition, IMT lets users unlock the full potential of their imagery by combining 3D analysis capabilities with their GIS data. The tool is specifically designed to make highly accurate vertical and horizontal 3D measurements of structures and surfaces, including calculating distances, areas, slopes, and azimuths in complex 3D features.
Baumgartner explained that oblique imagery is already used extensively in tax assessment and public safety applications, but many other markets are waiting to be served by off-nadir image data. The Icaros vision is to expand the use, visualization and analysis of oblique aerial images into the entire spectrum of GIS markets. Oblique analysis of structures can be beneficial to a host of new applications such as facility management, pipeline corridor infrastructure monitoring, energy audits, economic development, and asset management.
“Asset and facility management are the next big applications for oblique imagery within the emerging 3D GIS market,” said Baumgartner.
As GIS users continue the move to 3D, oblique imagery will be critical for constructing accurate image-based realistic 3D models. IMT enables users to manipulate and view oblique imagery while leveraging other geospatial data layers within their GIS environment. This capability allows users to navigate multi-image scenes in three dimensions while zooming and panning.
The IMT software also provides an optional Icaros Digitizer Tool (IDT) for generating 3D models. “IDT provides additional tools to extract physical building structures and digitize them into open format models, such as Collada, Obj, and Ply, textured from the source imagery,” Baumgartner said. “Automated 3D model generation from oblique imagery is at the intersection between imagery and GIS.”
Three weeks ago, GPS World / Geospatial Solutions held a webinar highlighting new technologies for imagery and data capture. The webinar had four presenters: Paul Smith of CycloMedia, Ted Ralston with Soft Power Solutions, Peter VanAmburgh from IIF Data, and John Ciampa CEO of Alta. You can view a YouTube video of the session. Because webinar time is limited, we couldn’t cover the technologies in detail, so I’m covering some of the technologies one column at a time.
In February, I devoted my column to CycloMedia, so you may want to review that material if you want more detail. This month I want to delve into the Alta balloon system that was presented by John Ciampa, the CEO of Alta. John was the original patent holder of Pictometry, the revolutionary high resolution geo-referenced metric oblique imagery system. He and Steve Schultz took the concept from theory to a practical functioning system that has been an industry standard for over 10 years.
John continues his research, dividing his time between the Rochester Institute of Technology (RIT) and Florida International University (FIU) while also working with the National Science Foundation (NSF). Although Pictometry was very successful, he also understood the limitations and cost of a manned aircraft as a capture system, especially for disaster response. Pictometry was a very capable system, but John felt that what was needed was a system that had a “lighter footprint” figuratively and literally. John took his knowledge and experience, and combined it with the latest developments in micro-miniaturized technology, to develop a system that was elegant in its simplicity and usefulness.
The Platform
Simply put, the Alta balloon is a steerable oblique geo-referenced camera system attached to a weather balloon. By using a balloon, John solved several problems associated with manned aircraft — cost, image quality, and accessibility. A balloon is cheaper than an aircraft, doesn’t require a licensed pilot, is more easily deployable, and can “fly” at lower altitudes.
Even a small aircraft is expensive, and requires a trained pilot and complex support logistics. A small used aircraft can run $50,000 to several hundred thousand dollars. Additionally, most high-end aerial imagery systems generally require expensive FAA-approved modifications of the airframe. This adds cost, and limits the aircraft that can be used. Equipped with Pictometry cameras and electronics, a total aircraft capture system can easily cost $300,000 and up.
A significant factor during disaster response events is transportability and support. Past experience has shown that it can be very difficult to transport and operate even small aircraft in disaster regions, especially if the damage is widespread. Fuel and ground support in disaster sites can also be a serious limitation. By comparison, a balloon system can be carried in a suitcase, shipped quickly, and set up in less than an hour. Since the cost is in the range of several thousand dollars, multiple systems are practical and can be deployed in numerous remote locations. Equally impressive is that operators can be trained in less than an hour.
But don’t get the impression that Alta is as simple as strapping a digital camera to a balloon. Achieving the image quality, accuracy and dynamic performance of the Alta system requires a very sophisticated package, including the balloon, controls, communications and sensors. The balloon system is actually two balloons, one within the other, an outer balloon and inner lift gas balloon. The outer balloon provides some external protection while presenting a consistent profile. The inner balloon provides the lift and is filled with either hydrogen or helium.
The altitude is remotely controlled by venting lift gas to descend or by dumping water ballast to ascend. An onboard computer and sensors can also maintain altitude autonomously. When the balloon is brought down, a tethered weight drops to several feet under the balloon. The weight contacts the ground first and “anchors” the balloon and payload for retrieval. This keeps the balloon and payload off the ground, and also makes it easier to spot.
There are several modes of operation that are determined by the operator prior to launch:
“Path Mode”: The balloon is released, ascending to the programmed altitude, then drifting with the wind currents before descending back to the ground.
“Patch Mode – single tether”: Used to launch and retrieve the balloon. The balloon ascends to altitude, and its position is downwind based on the strength of the wind acting on both the balloon and tether.
Patch mode with single tether.
“Patch Mode – multiple tethers”: Very precisely controls the location of the balloon over a limited area. This is very similar to the overhead cameras used in televised football games but in reverse, since the balloon wants to fly up.
Patch mode with multiple tethers.
The system can also be equipped with a parachute for emergency landings, a solar trickle charger for extended missions, and even a quadcopter that can steer the balloon to specific target areas. The lift capability of the balloon permits significantly longer duration flights than a quadcopter alone. A detailed operator’s manual is available for review at the Alta website — look for the Operators Manual.
The Payload
The modular payload is complex, but also lightweight and compact thanks to the latest developments in miniaturization. It consists of a precision GPS unit, inertial measurement/navigation unit (IMU), an onboard computer, environmental sensors, Wi-Fi communications, and an aimable high-resolution camera on a stabilized gimbal. The camera can be RGB, night vision or even multi-spectral. The imagery is downloaded as captured and delivered almost real-time.
Modular payload.
The Output
Because the balloon floats at low altitudes, image resolution is an impressive “game changer.” Here is just one example comparing a 4-inch pixel Pictometry/Bing image taken from 3,000 feet to a 1-centimeter pixel Alta image of the same location taken from several hundred feet.
Bing (left) and Alta images of stadium seats, compared.
Similar to Pictometry, the system uses GPS and IMUs to very accurately determine the camera location and attitude. That information, linked via algorithms to the captured imagery, results in imagery that is geo-referenced, measurable and available with full metadata, including the time of capture. I can’t over emphasize how important this is. Having instant access to imagery is nice, but having that imagery already geo-referenced means that the imagery can be quickly and easily imported into a GIS and overlaid with legacy GIS data for instant analytics.
The system has been used for many mundane applications such a real estate, agriculture, construction/engineering and event planning. More critical applications include crime-scene monitoring, surveillance and disaster response. Recently, in response to a South Florida Mall shooting, police launched an Alta balloon to view the crime-scene location. The imagery was instantly and continuously sent to police station computers and mobile devices of responders en route to the mall for pre-planning of their response. Viewing rooftops and walkways for victims and perpetrators, a near real-time operational picture was provided to police before putting themselves and others in harm’s way.
Actual image from an Alta balloon used in SWAT team maneuvers.
Dolphin Mall Sweetwater Florida, May 14, 2014, 6:05 a.m.
The Potential
I’m especially excited about the potential lifesaving use of Alta balloons. On numerous occasions I was involved in emergency response actions, and the dominant overarching need was high-quality imagery that could be combined with legacy data and imagery as close to real-time as possible. This system answers that need, and at a low cost. I could envision several Alta systems in every county nationwide ready to deploy on a moment’s notice. I believe that these units would be especially valuable for disaster response in second- and third-world countries. Dozens of Alta systems and trained operators could be delivered on short notice to major disaster sites, providing almost real-time common operational pictures for first responders. The added advantage is the very light need for logistics and support.
Military applications could be equally important. The silence of balloons coupled with a small visibility profile, including almost total invisibility at night, makes them ideal for reconnaissance and surveillance. The relatively low cost of the platforms also permits them to be expendable. In a tethered mode, the persistent “eye in the sky” could serve as a deterrent, or at a minimum make hostile activity more complicated for the perpetrators as they try to hide activities from the balloon. The “light” logistics and fast operator learning curve are just added benefits.
A key question raised during the webinar was FAA control. John indicated that the FAA does not consider the Alta balloons in a tethered mode subject to their control, and is currently reviewing it in a drift mode. John further amplified that the very low altitude of operation and dual control of descent should also exempt the drifter from FAA involvement. Alta could provide a significant advantage where UAS operations are restricted or not practical.
In a recent book, “Smaller Faster Lighter Denser Cheaper” by Robert Bryce, reviewed in the Wall Street Journal, the author argues that a similar dynamic, making less do more, drives virtually every technological change that has created the modern world, from cars and airplanes to advanced medicine, strategic metals and the iCloud. Alta balloons are certainly a good example.
I was an early proponent of Pictometry because, unlike abstract GIS data and ortho imagery, the metric oblique imagery was easily understood by non-GIS users. I saw many examples where it saved lives because police and firefighters were able to form and exploit a common operational picture quickly. Alta has me equally excited because it brings that same capability to users with a much simpler system that delivers almost real-time imagery at a cost anyone can afford. This technology is going to help a lot of people.
Three weeks ago, GPS World / Geospatial Solutions held a webinar highlighting new technologies for imagery and data capture. The webinar had four presenters: Paul Smith of CycloMedia, Ted Ralston with Soft Power Solutions, Peter VanAmburgh from IIF Data, and John Ciampa CEO of Alta. You can view a YouTube video of the session. Because webinar time is limited, we couldn’t cover the technologies in detail, so I’m covering some of the technologies one column at a time.
In February, I devoted my column to CycloMedia, so you may want to review that material if you want more detail. This month I want to delve into the Alta balloon system that was presented by John Ciampa, the CEO of Alta. John was the original patent holder of Pictometry, the revolutionary high resolution geo-referenced metric oblique imagery system. He and Steve Schultz took the concept from theory to a practical functioning system that has been an industry standard for over 10 years.
John continues his research, dividing his time between the Rochester Institute of Technology (RIT) and Florida International University (FIU) while also working with the National Science Foundation (NSF). Although Pictometry was very successful, he also understood the limitations and cost of a manned aircraft as a capture system, especially for disaster response. Pictometry was a very capable system, but John felt that what was needed was a system that had a “lighter footprint” figuratively and literally. John took his knowledge and experience, and combined it with the latest developments in micro-miniaturized technology, to develop a system that was elegant in its simplicity and usefulness.
The Platform
Simply put, the Alta balloon is a steerable oblique geo-referenced camera system attached to a weather balloon. By using a balloon, John solved several problems associated with manned aircraft — cost, image quality, and accessibility. A balloon is cheaper than an aircraft, doesn’t require a licensed pilot, is more easily deployable, and can “fly” at lower altitudes.
Even a small aircraft is expensive, and requires a trained pilot and complex support logistics. A small used aircraft can run $50,000 to several hundred thousand dollars. Additionally, most high-end aerial imagery systems generally require expensive FAA-approved modifications of the airframe. This adds cost, and limits the aircraft that can be used. Equipped with Pictometry cameras and electronics, a total aircraft capture system can easily cost $300,000 and up.
A significant factor during disaster response events is transportability and support. Past experience has shown that it can be very difficult to transport and operate even small aircraft in disaster regions, especially if the damage is widespread. Fuel and ground support in disaster sites can also be a serious limitation. By comparison, a balloon system can be carried in a suitcase, shipped quickly, and set up in less than an hour. Since the cost is in the range of several thousand dollars, multiple systems are practical and can be deployed in numerous remote locations. Equally impressive is that operators can be trained in less than an hour.
But don’t get the impression that Alta is as simple as strapping a digital camera to a balloon. Achieving the image quality, accuracy and dynamic performance of the Alta system requires a very sophisticated package, including the balloon, controls, communications and sensors. The balloon system is actually two balloons, one within the other, an outer balloon and inner lift gas balloon. The outer balloon provides some external protection while presenting a consistent profile. The inner balloon provides the lift and is filled with either hydrogen or helium.
The altitude is remotely controlled by venting lift gas to descend or by dumping water ballast to ascend. An onboard computer and sensors can also maintain altitude autonomously. When the balloon is brought down, a tethered weight drops to several feet under the balloon. The weight contacts the ground first and “anchors” the balloon and payload for retrieval. This keeps the balloon and payload off the ground, and also makes it easier to spot.
There are several modes of operation that are determined by the operator prior to launch:
“Path Mode”: The balloon is released, ascending to the programmed altitude, then drifting with the wind currents before descending back to the ground.
“Patch Mode – single tether”: Used to launch and retrieve the balloon. The balloon ascends to altitude, and its position is downwind based on the strength of the wind acting on both the balloon and tether.
Patch mode with single tether.
“Patch Mode – multiple tethers”: Very precisely controls the location of the balloon over a limited area. This is very similar to the overhead cameras used in televised football games but in reverse, since the balloon wants to fly up.
Patch mode with multiple tethers.
The system can also be equipped with a parachute for emergency landings, a solar trickle charger for extended missions, and even a quadcopter that can steer the balloon to specific target areas. The lift capability of the balloon permits significantly longer duration flights than a quadcopter alone. A detailed operator’s manual is available for review at the Alta website — look for the Operators Manual.
The Payload
The modular payload is complex, but also lightweight and compact thanks to the latest developments in miniaturization. It consists of a precision GPS unit, inertial measurement/navigation unit (IMU), an onboard computer, environmental sensors, Wi-Fi communications, and an aimable high-resolution camera on a stabilized gimbal. The camera can be RGB, night vision or even multi-spectral. The imagery is downloaded as captured and delivered almost real-time.
Modular payload.
The Output
Because the balloon floats at low altitudes, image resolution is an impressive “game changer.” Here is just one example comparing a 4-inch pixel Pictometry/Bing image taken from 3,000 feet to a 1-centimeter pixel Alta image of the same location taken from several hundred feet.
Bing (left) and Alta images of stadium seats, compared.
Similar to Pictometry, the system uses GPS and IMUs to very accurately determine the camera location and attitude. That information, linked via algorithms to the captured imagery, results in imagery that is geo-referenced, measurable and available with full metadata, including the time of capture. I can’t over emphasize how important this is. Having instant access to imagery is nice, but having that imagery already geo-referenced means that the imagery can be quickly and easily imported into a GIS and overlaid with legacy GIS data for instant analytics.
The system has been used for many mundane applications such a real estate, agriculture, construction/engineering and event planning. More critical applications include crime-scene monitoring, surveillance and disaster response. Recently, in response to a South Florida Mall shooting, police launched an Alta balloon to view the crime-scene location. The imagery was instantly and continuously sent to police station computers and mobile devices of responders en route to the mall for pre-planning of their response. Viewing rooftops and walkways for victims and perpetrators, a near real-time operational picture was provided to police before putting themselves and others in harm’s way.
Actual image from an Alta balloon used in SWAT team maneuvers.
Dolphin Mall Sweetwater Florida, May 14, 2014, 6:05 a.m.
The Potential
I’m especially excited about the potential lifesaving use of Alta balloons. On numerous occasions I was involved in emergency response actions, and the dominant overarching need was high-quality imagery that could be combined with legacy data and imagery as close to real-time as possible. This system answers that need, and at a low cost. I could envision several Alta systems in every county nationwide ready to deploy on a moment’s notice. I believe that these units would be especially valuable for disaster response in second- and third-world countries. Dozens of Alta systems and trained operators could be delivered on short notice to major disaster sites, providing almost real-time common operational pictures for first responders. The added advantage is the very light need for logistics and support.
Military applications could be equally important. The silence of balloons coupled with a small visibility profile, including almost total invisibility at night, makes them ideal for reconnaissance and surveillance. The relatively low cost of the platforms also permits them to be expendable. In a tethered mode, the persistent “eye in the sky” could serve as a deterrent, or at a minimum make hostile activity more complicated for the perpetrators as they try to hide activities from the balloon. The “light” logistics and fast operator learning curve are just added benefits.
A key question raised during the webinar was FAA control. John indicated that the FAA does not consider the Alta balloons in a tethered mode subject to their control, and is currently reviewing it in a drift mode. John further amplified that the very low altitude of operation and dual control of descent should also exempt the drifter from FAA involvement. Alta could provide a significant advantage where UAS operations are restricted or not practical.
In a recent book, “Smaller Faster Lighter Denser Cheaper” by Robert Bryce, reviewed in the Wall Street Journal, the author argues that a similar dynamic, making less do more, drives virtually every technological change that has created the modern world, from cars and airplanes to advanced medicine, strategic metals and the iCloud. Alta balloons are certainly a good example.
I was an early proponent of Pictometry because, unlike abstract GIS data and ortho imagery, the metric oblique imagery was easily understood by non-GIS users. I saw many examples where it saved lives because police and firefighters were able to form and exploit a common operational picture quickly. Alta has me equally excited because it brings that same capability to users with a much simpler system that delivers almost real-time imagery at a cost anyone can afford. This technology is going to help a lot of people.
