Industry experts noted in our November 2022 issue that heavy equipment autonomy may be a distant future. However, the steady innovation in machine-control technology to get there is yielding substantial value. To drill deeper into those technologies, we interviewed additional industry experts with a focus on the key role of GNSS in such systems.
1D, 2D and 3D
There is currently a sharp growth in the adoption of 3D systems, according to Jordan Van Wie, product specialist with SANY America, a prominent manufacturer of construction equipment. “The fact is that many jobs are requiring this. They’re more efficient in their bidding process. They know exactly where they need to cut and where to fill — this means being more productive in less time.”
SANY America is based in Peachtree City, Georgia, where many of its construction equipment systems are manufactured, including the SY225C, a popular medium excavator.
The process of automating to the levels the operators desire is a matter of which sensors are added and how they sense the active geometry of the equipment in use.
For an excavator, SANY installs four sensors, then measures the machine, said Mukesh Selvaraj, product manager, medium and large excavators, SANY America.
“We know the distance between the bucket pin and the stick pin, up through the boom, and the angles on the sensors. We can compute in the system and report where the tip of the bucket is in relation to the body, and construct a 3D model in real time. This reporting can be as fast as 200 Hz.”
Among the machine-control systems implemented on SANY construction equipment are those from Hexagon | Leica Geosystems. Leica produces precision guidance and control sensors and systems for construction, agriculture and mining that are integrated onto various heavy equipment brands.
While 3D is becoming more popular, systems need to be scalable. Hexagon | Leica Geosystems has variants for different levels of guidance and automation, said Kert Parker, U.S. channel development manager for the company.
“For instance, if you start with our PowerDigger Lite, it has a control box, a display, a boom sensor, an angle sensor for the stick (which includes a laser catcher) and a 360° bucket sensor. This lets you know where the bucket tip is in relation to the model — call it a 1D system.” The cost of such a system might only be 5% or 10% of the cost of the machine on which it is installed — a modest investment for the productivity gains it can deliver.
To upgrade and run automatics, users could add a machine control panel and docking station with just 2D software. “That will give you a semi-automatic solution even on 2D. Then you can upgrade and add the GNSS receiver and antenna — or antennas — and 3D software to make it 3D, semi-automatic,” Parker said.
Two-thirds of the price of the base system is for the sensors on the boom, stick, bucket, the pitch and roll sensor, and the wires that communicate throughout the system, Parker explained.
“So, it’s completely scalable. You can start with a low-cost system upgrade to do GNSS fully and semi-automatic. We can automate any pilot-controlled machine, then we set the pressure. And when we sense the stick pressure, if the system is going automatic, then we automate the boom and the bucket.”
Third ‘D’ Options
“When you’re using something to give the machine a northing, an easting and a height at all times — that is when it becomes full 3D,” Parker said.
3D systems can be configured with a single GNSS receiver, with dual GNSS receivers, or off of a robotic total station. “The only difference between single and dual is that, with single, every time you move the machine you have to do a calibration swing, about 90° to get your heading again.”
“You can dig curves and complex designs working in 2D,” Van Wie said. “But every time you move the machine, you have to re-bench to a known reference, either by pinching with a bucket’s teeth, or hit the stick sensor that has an incorporated laser catcher. When you move the machine, you catch the laser beam again, and you use that for your known reference to dig back from the 3D model.”
For certain operations — such as excavating in a straight line or moving materials to the side —higher levels of automation may not be needed, so some users appreciate the option of starting with a cheaper system.
“For the small operator, of course, but even for a large operator, it’s a big investment to go full 3D,” Van Wie said. “They don’t want to go full 3D right away, or not on all equipment at once. They start off with just the basics and get familiar with it. Then when they want to upgrade, they have some of the stuff that they’re going to need for their machine already on it.”
System Examples
eSurvey GNSS manufactures GNSS-based equipment, software and systems for surveying, mapping, agriculture, UAV and construction. Better known in other global markets than in North America, the company has seen a steady rise in the market for construction automation — outpacing other sectors utilizing heavy equipment automation such as agriculture and mining combined. For construction, in many parts of the world excavators are the prime focus for automation.
Their eME10 system for excavators includes a dual-antenna GNSS receiver, three single-axis tilt sensors, one dual-axis tilt sensor, a tablet and software (Figure 1). “The eME10 does not support a rotating bucket at this time,” said Edward Zhang, product manager for machine control technology. “We support standard excavators, excavators that reach into the water (for instance on dredging barges), and with different bucket tools such as quartering hammers and milling tools.”
Another popular system for compactors is the eMC10, with a single-antenna GNSS receiver, tablet and software, and optional temperature and vibration sensors.
Managing Positioning
High-precision GNSS, as implemented for architecture, engineering and construction (AEC) applications, can yield centimeter-grade results. However, as many AEC professionals and practitioners know, achieving repeatable and consistent results requires an experienced and skilled GNSS operator. Is the operator examining the results for statistical consistency? How have the observations been constrained to the desired reference framework? Have sources of error such as multipath and space weather been considered?
However, Nick Fifarek, general manager at SITECH Pacific LLC, a construction technology provider, said that equipment operators only need to learn the user interface.
“They are mostly concerned with how the grade is shown in the model, and what actions are required to meet the grade. They should not need to be concerned with the working of the GNSS receiver.”
A larger firm with multiple systems will usually have a technician or surveyor on board, Fifarek explained. This expert would have the experience needed to set up a GNSS site base, ensure corrections are received, and troubleshoot causes of anomalies and poor results.
To be efficient, an operator should not have to deal with a complex set-up.
“It should be more like Google maps in your car,” Fifarek said. “They do not need to know how the model was created, and how the GNSS delivers positions to the interface. All the sensors should work seamlessly, like tilt sensor and IMUs [inertial measurement units] and how they work together with the GNSS to put positions on the blade or bucket. Once this is all working well and the model is applied, they should just be able to take directions.”
Nevertheless, sometimes this expert will need coaching, or a small firm may not have an expert at hand.
“We may need to teach them about some fundamentals, such as signal-to-noise ratio, PDOP [positional dilution of precision], and other quality indicators — especially when setting up the site base station,” Fifarek said.
Additionally, he pointed out, the control must be set up — this is mostly done by engineering or surveying firms along with site calibrations — and operators need to know how to check it.
Multipath Issues. Fifarek has not experienced problems with short masts for GNSS antennas, saying that the height of the cab is sufficient. Modern multi-constellation receivers, have improved multipath mitigation, and are able to work in sites with limited sky view or obstructions. Equipment such as excavators and dozers typically have dual-antenna GNSS systems, or two receivers and antennas. This provides not only position, but orientation and heading. These are usually installed on the body or cab, although some systems have a GNSS antenna on each end of the blade. Some systems use a method that only fixes one of the antennas/receivers, and then performs a fixed baseline solution for orientation.
The Chain of Components
Much like autonomy in vehicles, machine control implementation can be defined as various levels.
Level 1: GNSS-assisted guidance. The most basic level of implementation provides the equipment’s location and heading. It acts the same way as a navigation device or phone in your car. The technology has been around for decades for precision agriculture and construction.
Level 2: Implement Control. Control of the blade or bucket.
Level 3: Assist. Implement control plus a level of automation where the operator moves the control stick to initiate an action the machine completes by moving the blade or bucket to meet the design model geometry. This can include steering for various types of equipment.
Level 4: Autonomy. More on that later.
For levels 2 through 4, continuously updating a position on the blade or bucket requires a chain of sensors to work in tightly controlled harmony. An excavator could be equipped with one or two GNSS receivers and antennas and a tilt sensor on the body, explained Geoffrey Kirk, product manager, autonomy and assist for Trimble. The GNSS will provide the position and orientation of the body, or rotating section of the body, on an excavator, and the tilt sensor reads how level it is. Another option is positioning with a total station and prism on the body, such as when GNSS is not available. “Either way, you need to know where you are in 3D space to be able to work on any 3D model,” Kirk said. “Today there are usually about 30 satellites in view. We can do so much more now compared to the days when we had fewer satellites, things that would have been impractical,” Kirk continued.
Sensors on the boom, stick and bucket can be likened to an upper arm (boom), forearm (stick) and hand (bucket), with rotating buckets acting like a wrist.
“We put a six-degrees-of-freedom IMU at each of these locations,” Kirk said. This is a chain of highly dependent geometry extended out to the bucket. However, Kirk said there may be a better way.
Reducing the Links
In recent years, a new technology has been implemented for GNSS smart antennas (rovers), like those that surveyors and grade checkers use, which tightly couples IMUs and movement of the GNSS antenna for calibration-free tilt compensation. Examples include the Trimble R12i (for surveying) and R780 (for construction), Leica GS19 T, and many more — few high-precision rovers made today lack tilt compensation. The observed acceleration and direction of the antenna adds orientation to the tilt angle (from the onboard tilt sensors), so the position of the tip of the survey rod can be computed precisely and in real time.
At the Bauma construction trade fair held in November 2022 in Munich, Germany, Trimble gave participants a peek at something new: putting a tilt-compensating GNSS smart antenna out on the stick of an excavator.
“With current systems, every time you hit one of those joints on an excavator, you need to understand what it is doing, calculating angles along the way,” Kirk said. “By mounting a tilt-compensated GNSS receiver on the stick, this becomes a lot easier to do.” Such innovations dovetail well with another trend in construction equipment: a move from purely hydraulic steering to drive-by-wire. This trend makes for more simplified and often less costly processes for adding implement control and automatics, but may also be key in implementing autonomy.
The Path Toward Automation
“One of the big changes in the industry is understanding what tasks operators are trying to do, so that we can help them do those tasks,” said Kirk. “We want to help people be more productive. We know autonomy is a thing. We’re actively working on autonomy; it’s going to be a while. In the interim, we want to make sure that we are providing value to the manual operators for the tasks that we can’t do autonomously.”
Key foundational components of what would go into autonomous systems are already in place.
“With automatics, you already have implement control, and in some implementations, you even have steering,” Kirk said. “What is missing in terms of the mechanics is speed control — that may be the easy part.” Adding the crucial situational awareness, other sensors for feedback, and the brains for automation is what might take a lot of time to work out.
“Autonomy for cars is where you are trying to avoid hitting things,” said Kirk. “For construction, we are in the business of hitting piles of dirt and spreading them around.” For a car, the sensors see something, recognize it, know how far away it is, and can issue such commands as “stop” or “slow down” — which is not so simple for construction.
Three key technologies you’ll see being used for situational awareness are radars, cameras and lidar, mostly used in combination. “Radars have some really nice behaviors,” explained Kirk, but cautioned that they cannot tell what they are doing.
For instance, adaptive cruise control in cars, which is nearly always done with radar, works very well and reliably. Most such radars are now solid state and safety certified. Unfortunately, he points out, while radar is very good at alerting drivers that there is something in front of them, it is not very good at telling them what it is.
“That’s why developers put in cameras, so that you can see whether what’s in front of you is a person, another vehicle, or something else. That’s why you have those combinations of sensors.”
One of the reasons it will take longer to automate construction, Kirk explained, is that operators need to know much more about the nature of other objects in the construction environment than cars do on the road. The operators need to know not only what people, equipment and materials are around them, but also whether there is something or someone standing in front or on top of the pile of dirt.
“For situational awareness, you need to be able to do real-time mapping,” Kirk said. “Lidar and cameras, such as stereographic cameras, can be used as classifiers. Lidar can have limitations, such as when driving directly into the sun.”
“The smarts for autonomy are knowing what the task is and how to perform that task,” Kirk said. “However, from the standpoint of a machine’s sensor and setup, we’re not controlling speed, though we do on agricultural machines. So, machines are matched really well for autonomy — you can make them do whatever you want today.”
Examples of autonomous conduction systems were demonstrated in the off-site “sandbox” exhibit of Trimble Dimensions+ held in November 2002 in Las Vegas. There was an autonomous excavator, a compactor and a remote-control dozer.
Yet these were operating in a controlled environment. Kirk said that for safety reasons, early adoptions of autonomy might be confined to sites that are not along roads and highways.
Read more of this cover story, “Why GNSS is the glue for construction.”
