Komatsu plans to introduce NVIDIA graphics processing units (GPUs) to its SmartConstrution jobsites. The GPUs will communicate with drones from Skycatch, a Komatsu partner, which will collect 3D images, generate terrain data and “visualize” site conditions.
Komatsu is deploying the artifical intelligence (AI) project as an extension of its SmartConstruction initiative in Japan; the drone-assisted, automated equipment service was launched to alleviate the burden of the country’s severe shortage of skilled workers.
The company has deployed SmartConstruction at than 4,000 jobsites across the country, and the AI extension will be integrated into those sites.
Working with NVIDIA, OPTiM Corp. — another Komatsu partner and an internet of things management software company — will provide an application to correlate terrain data to jobsite workers and construction machines for visualization.
Enter Jetson. At the center of this collaboration is the NVIDIA Jetson artificial intelligence platform. When Jetson, which works with NVIDIA’s cloud technology, is installed in construction machines, it will be able to provide 360-degree images, enabling prompt recognition of workers and other machines nearby. The technology could potentially decrease fatalities that result from workers being struck by an object, piece of equipment or vehicle.
Jetson will also be used with the stereo cameras installed in the cabs of construction equipment, and will recognize continuously changing jobsite conditions on a real-time basis, to better provide accurate instructions to machine operators.
Future plans call for use not only for automatic control of devices, but also for high-resolution rendering and virtual simulation of construction and quarry jobsite operations.
There was plenty for a philosophy major to sink his teeth into at ION’s January workshop on Cognizant Autonomous Systems for Safety Critical Applications (CASSCA).
What is knowledge? What is meaning? What is understanding? What is intelligence? What is learning? What is thinking?
These questions excited Plato and Kant, Buddha and Descartes, perhaps out of intellectual or spiritual curiosity. Who’s to say? But the people asking them now are driven, quite literally, by practicalities. They have come to realize that we cannot ride in driverless cars or fly in pilotless plane-taxis, we cannot live in an autonomous, artificially intelligent environment without knowing a bit more exactly what knowledge is, in this brave new world.
Without thinking about what thinking may be, for a machine.
Why does this matter to a GPS/GNSS/PNT readership? Because as positioning and navigation engage more deeply with artificial intelligence (AI) generally, and with autonomy in particular, these issues emerge as part of the environment that such solutions explore, and in which they must verify and validate themselves.
Welcome to the future, it’s yours. Now think about it.
Culture Club. Some of us may have believed that only technical obstacles remain in the path of a driverless car and an otherwise automated society, salted with a few regulatory wrinkles to iron out. But as build-a-robot R&D projects transform into full commercial partnerships, cultural challenges jump up as well: inertia, instability of requirements, unanticipated expectations, magical thinking (the development of empathetic attitudes towards robots), misplaced trust and misplaced distrust. All this according to Signe Redfield, roboticist and mission manager at the U.S. Naval Research Laboratory.
Joao Hespanha, professor of electrical and computer engineering at the University of California, Santa Barbara, outlined three key concepts for AI development: computation, perception and security. The critical questions for the first named are, how much computing will be done onboard the platform, how much learning will be done onboard, and how much of each process will be distributed to offboard computation. Perception, a crux for autonomy, is closely bound in a feedback loop with control. The platform must gather data to make autonomous decisions (control), and those decisions must maximize the gathering of information (perception).
Amply consider security. All safety-critical systems must provide for — and prevent where possible — decisions based on compromised measurements, which may stem from system or environmnetal noise, sensor faults, hacked sensors, or other corruptions.
Second Wave. We are in the second wave of AI, according to Steven Rogers, senior scientist for sensor fusion at the Air Force Research Laboratory. In the first wave, 60s and 70s, large and complex algorithms, relatively low on data, drove new developments — but they hit real-world problems, hard. Since the mid-80s, we have been in the “classify” stage with relatively simpler programs generating and consuming lots of data. Intense statistical learning will eventually lead to the third wave of AI: Explain.
On a timeline yet to be determined, contextual adaptation will give rise to “explainable” AI, capable of answering unexpected queries. That is, it will have learned how to teach itself.
Some of this stuff gets pretty scary.
Most future knowledge will be machine-generated.
Let’s run through that one more time.
“Most future knowledge on Earth will come from machines extracting it from the environment,” said Rogers. “Machine generation of knowledge is key for autonomy.”
Here’s where the thought processes really started to levitate. “Current sense-making solutions are not keeping pace, not growing as knowledge is growing,” Rogers asserted. And he challenged us with the questions posed at the beginning of this column: in AI, the context we will use to explore much of the future, what is knowledge? What is meaning? And so on.
He gave us one of his answers: “Knowledge is what is used to generate the meaning of the observable for an autonomous system. Correspondingly, machine-generated knowledge is what is used to turn observables into machine-generated meaning.”
Slide from Steven “Cap” Rogers’ presentation at CASSCA.
He suggested a book by George Lakoff and Mark Johnson, Metaphors We Live By. Pretty heady stuff for a room full of engineers. I don’t know about you. I’m headed down to the library to check it out.
Requirements, Simple/Not. We got back to earth with some technical challenges we could actually chew on with David Corman, program manager for Cyber-Physical Systems and Smart and Connected Communities at the National Science Foundation. Seemingly simple requirements for safety-critical applications break down into hundreds of requirements that no one has really thought about, Corman said, as he displayed a chart of “Some Example Research Problems.”
Precision agriculture and environmental monitoring are two sectors where he thought autonomous operations come closest to being full realization, because their operational environments are structurally defined enough. In such constrained niches that we more fully understand, we can implement autonomous operations. Elsewhere, “we don’t know how to specify what we want, so that we get only ‘good results’ and no ‘bad results.’ ”
He identified a looming Cambrian explosion in AI, analogous to that for plants and animas following the dinosaur extinction, in which systems interact, gather data, sense the environment, learn, improve and multiply. He suggested we browse “The Seven Deadly Sins of Predicting the Future of AI,” an essay by Rodney Brooks.
The afternoon’s workshop talks followed, from experts in autonomous flight software, legal and insurance aspects of autonomy, the Ohio State University’s Center for Automotive Research, and the U.S. Department of Transportation. But I tell you, this morning done my brain in.
Before folding up, I must mention a short video on autonomous flying taxis displayed by Paul DeBitetto, VP of software engineering at Top Flight Technologies. It depicts Pop.Up, a modular ground and air passenger vehicle for megacities of the future. Check it out.
The CASSCA workshop was organized and moderated by Zak Kassas, an assistant professor at the University of California, Riverside and director of the Autonomous Systems Perception, Intelligence & Navigation (ASPIN) Laboratory. He is also co-author of two cover stories in GPS World, “LTE cellular steers UAV” and “Opportunity for Accuracy.”
ION president John Raquet expressed the hope that we may see a fully fledged conference on this topic in the near future: CASSCA 2019, perhaps, to join the rotating repertory of ION annual meetings.
Agreed. We need to think more.
Don’t look back, the machines may be gaining on us.
A conference aimed at military and government agency decision-makers is scheduled for Feb. 6–8.
AUVSI Unmanned Systems – Defense. Protection. Security. (USDPS) will highlight what’s next for the defense, protection and security industries. The conference has been reimagined to include more opportunities to learn from industry and government thought-leaders and shop for the latest technology, said AUVSI.
Expanded focus beyond the military includes government agencies such as Homeland Security, Interior, Justice, Energy, the FBI and NOAA, as well as other public safety agencies.
Thought leaders and subject matter experts from the military, government agencies and public safety outlets will provide details and insight on investments, innovation and sustainment, including what’s next with artificial intelligence and cybersecurity.
More than 50 companies will be on site to provide military and government agencies with an inside view of new technologies and trends.
In addition, a public safety educational lineup defines how to best use unmanned systems for explosive ordinance disposal and HAZMAT applications, addresses the counter-UAS threat at home and abroad, and provides insight on how disaster and emergency responders are using unmanned systems.
The conference will take place at the Gaylord National Resort & Convention Center in National Harbor, Maryland. Those registering before Nov. 19 can take advantage of advanced rates.
Can artificial intelligence fly a drone? Can a drone catch thermals the way birds do?
Microsoft researchers are partnering with the Nevada Governor’s Office of Economic Development (GOED) and the Nevada Institute for Autonomous Systems (NIAS) to find out.
The artificially intelligent UAS being tested at the Nevada UAS Test Site is a 16 ½ -foot, 12 ½- pound sailplane. The sailplane relies on a battery to run onboard computational equipment and controls such as the rudder, plus radios to communicate with the ground.
It also has a motor so that a pilot can take over manual operation when necessary.
But once it’s up in the air, the UAS demonstrated its ability to operate on its own, finding and using thermals to travel without the aid of the motor or a person.
Simple and complex UAS testing was conducted at the Hawthorne Advanced Drone Multiplex (HADM) Test Range located at Hawthorne, Nevada. HADM is a 230-square mile area where a variety of UAS applications can be tested, including artificial intelligence (AI).
NIAS manages the FAA-designated Nevada UAS Test Site, which includes HADM and other UAS test ranges across Nevada.
The Microsoft operation was based at the Hawthorne Industrial Airport where preliminary tests were made. Subsequent tests were conducted at an area east of Walker Lake around six miles from the airport.
The team flew three different sailplanes that reached an altitude of approximately 1,700 feet flying almost two dozen Nevada UAS Test Site Certification of Authorization (COA) flights Aug. 7-11.
“Innovative AI technology like what Microsoft tested with NIAS is clearly where the most dramatic global UAS Industry disruptions will occur,” said Chris Walach, test site director. “When you think of artificial intelligence or AI, there are many perspectives on the value-add to the UAS industry. Very evident to me, developing and testing AI, or machine learning technology, is going to have multiple applications that will significantly benefit the UAS Industry and the American way of life. This is one of the most exciting developments I have seen over the past several years in Nevada and globally.”
“Microsoft researchers have created a system that uses artificial intelligence to keep the sailplane in the air without using a motor, by autonomously finding and catching rides on naturally occurring thermals, like how wild birds stay aloft,” said Ashish Kapoor, a principal Microsoft researcher. “Birds do this seamlessly, and all they’re doing is harnessing nature and they do it with a peanut-sized brain.”
“Nevada wholeheartedly supports the growth of the Unmanned Aerial System industry, and teaming with global technology leader Microsoft to perform these Nevada-based tests speaks to our leadership role with the global community,” said Tom Wilczek, industry specialist for the Nevada Aerospace and Defense Industry for the Governor’s Office of Economic Development. “Governor Sandoval and our Legislature expect us to engage in the growth of transformative technologies and I am grateful for the opportunity afforded by Microsoft to team and to do just that.”
This e-newsletter marks the sun setting on the career of GeoIntelligence Insider columnist and contributing editor Art Kalinski, long active in geospatial circles. Art is handing the journalistic reins to William Tewelow, a former intelligence specialist with the U.S. Navy. He also served a special assignment at the U.S. Department of Transportation and is currently a manager with the U.S. Federal Aviation Administration (FAA). The following discussion muses about the direction of our geospatial tradecraft in the coming years.
Tewelow’s Geointelligence column will alternate monthly with Mike Jones’ Defense PNT column. Every issue of this newsletter also brings you the latest news on both expanding fronts, hardware and software, in the defense and security realm.
Passing the torch
A conversation between retiring GeoIntelligence editor Art Kalinski, GISP, and his successor, William Tewelow, GISP.
Counter-drone technology research and operation efforts are helping to prepare for the potential threat of drone swarms. This Lockheed Martin concept is explored in “Technology That Counters Drone Swarms.” (Image: Lockheed Martin)
Art: A factor of life is that you just can’t predict with certainty where technology is headed or what the issues will be. Several weeks ago I attended a UAV (unmanned aerial vehicle) users group that seemed like a glimpse of the Wild West. I thought the key discussion topics would be the latest technology which seems to change almost weekly. That was not the case. The key discussions were the same as the early days of GIS: legal liability, jurisdictions, data ownership, administrative issues and so on, as the FAA changes the rules and regulations. I began to see that there will be more employment opportunities for lawyers than techies and pilots.
Although most of the attendees at the conference were geospatial/first responder users, I’ve learned of a growing community of non-traditional users such as power line inspectors and even tower painters. Yes, a tower painting company using tethered UAVs to paint in dangerous locations such as bridges, tanks and towers. At first I thought that would be difficult but by using a tethered system, heavy batteries are eliminated and bigger UAVs will permit heavier hoses, etc. Although the FAA had little interest in tethered balloons and UAVs, that could change as the size, altitude and potential crash radius expands.
This is just one example of how our technology is moving in many unplanned directions, everything from UAVs with new ways of collecting imagery to more exotic topics such as artificial intelligence, machine learning, augmented reality, virtual reality (VR) and gamification. The entire trade craft is evolving in ways I don’t think anybody could have predicted just a few years ago, much less a decade ago. It will be interesting to see where it goes.
William: You mentioned several new technologies mixing things up. Small sats are also making a significant impact. They are game changers. Imagine the traditional 2D satellite imagery base map. With the tech we have now 3D base maps are possible and have been for a long time with DEMS. Now, small sat imagery can be overlaid atop the 3D basemaps providing fresh imagery with periodicities of 24 hours or less, including multi-spectral. I’ve even seen small sat video overlays. That’s game changing.
Art: Not only small sats but the work being done with drone swarms is an exciting new area. If data needs to be collected in a hostile area or if flight clearance in a busy air corridor was limited, a swarm of maybe 100 drones could capture a wide swath of data at high resolutions, with multiple spectrums in very short order.
Virtual reality and augmented reality are two areas which have always intrigued me and they are coming to fruition. When I started out in GIS, I told people working for me to do the best we could building our regional database because at some point we would have construction workers wearing goggles that will augment the reality of what they are seeing with GIS data. That day is here and people are finally starting to do that. So, the entire concept of AR (augmented reality) and VR I find extremely exciting.
William: Yes, absolutely, in fact I believe they used drone swarms in the hostage situation in Algiers in 2014. And you are right. VR and AR use for first responders is especially critical. Say you’re a disaster relief person at night in a smoke-filled building. Night vision and thermal vision reach a practical limit. However, combining the vision devices with AR you may be able to navigate through the environment well enough. AR shows at least what should be there so accurate and timely data is critical. I can see this is where things are headed. Additionally, more mundane tasks such as design and modifications of structures could be enhanced. Work is already being done using AR and VR to facilitate the design process.
Art: VR started as fun and games but to me serious use VR is where the new and exciting action will be for geospatial practitioners. Virtual trainers I saw recently included small arms and artillery trainers for the military to a less exciting but equally impressive virtual trainer to teach arc welding and even spray painting a car. Significantly more critical was a VR surgical trainer where the muscle learning and hand eye coordination required is significant. It’s obviously quicker, easier and cheaper to practice virtually than on a real human or real equipment and the VR environment is just one short step away from surgical robots operating in remote locations.
William: Right, and you are touching on gamification, where VR can be used to train toward certain results. Take the military, for example, gamification can save all manner of resources – bullets, people, buildings the environment. Additionally, users can also scale up or down. The VR environment doesn’t just mean roads, maps or mountains. You can get down to a micro level.
Art: It’s interesting you mention that. One aha! moment I had years ago was at the first Esri User Conference in San Diego. I went through the map gallery. A lot of examples of different uses for GIS were on display and one person had created a map of the human circulatory system using ArcView Network Analyst. Looking at it I thought, of course, whether its blood vessels or interstate highways, it’s a network; so, this individual was using a geospatial tool to map the entire blood vessel network.
William: We are now in the realm of nano technology, where things are small enough to work on blood cells. Imagine steering straight towards a cancer cell by pinpointing a location in relation to the body.
Art: It’s almost like the movie “Fantastic Voyage” except in a virtual environment.
William: Yes. Being able to put the nano technology right where it needs to be means entering an age of focused medicine so that a pill releasing nano particulates doesn’t impact the whole body just the area needed. That’s another discussion in itself. Of course, all of that will have to be controlled, mapped and understood in terms of what it’s doing and where it’s going in relation to the body.
Art: One area that is very concerning to me is cyber. All of this stuff we’ve talked about is very vulnerable whether stealing data or doing damage to a society. Thwarting those threats is getting into machine learning, AI and other topics bordering science fiction.
William: Yes, and you are getting into artificial intelligence. IBM, Google, Amazon, Facebook, Microsoft and our three letter agencies are all investing into the world of Geospatial AI. I am not sure where GeoAI will end up but we can already see some hints. The amount of geospatial data is overwhelming and imagine the explosion happening with IoT. How will all that be managed? GeoAI will do pattern recognition so in effect GeoAI is assisted intelligence.
As an example let’s consider a high profile event. There are 1000’s of parameters that need to be tracked such as surveillance cameras, social media feeds, a suite of sensors, etc. Identifying critical patterns is crucial so Geospatial Assisted Intelligence can monitor the parameters of the event then notify the analyst when a triggering event occurs. That’s where the technology is going where it has to go.
Art: One has to have a truly dizzying intellect to keep track of all aspects of our technology. At the last NGA (National Geospatial-Intelligence Agency) conference, Director Cardillo talked extensively about open systems and open sources which are growing directions for the agency. I believe it’s good to get away from everything being “inside the tent” and taking advantage of the extensive capability in the broad geospatial community. The stated goal is to be able to develop needed geospatial tools, in an unclassified environment, not in months or weeks but literally in minutes to provide those tools to analysts in the field. Things have come so far from the day I was just happy to get a pen plotter to print a simple map.
William: It really is mind boggling. I was there when full motion video began and that was dynamic 2D imagery but then it made the jump to 3D dynamic imagery and my circuits fried. I still have a hard time comprehending all the data crunching that goes into it and it is now almost standard. The future is amazing. That is the direction I want to continue to explore.
