NEC's autonomous backhoe: connecting robotics to the tough stuff

• Where is it best to start applying robotics in an economy?
• In Japan one of the sweet spots is the construction industry where skills are in short supply
• A trio of Japanese companies is working to roboticise heavy machinery, starting with the backhoe

Labour shortages due to the aging of skilled workers and a decline in the number of younger workers has become an urgent issue for the Japanese construction industry. Robotics looks like being about the only way forward, given that Japan rules out immigration as a way to rebalance its demographic.  So the Japanese tend to turn to labour saving measures to improve productivity - if you don't have enough skilled personnel you find ways to physically augment the ones you do have so they become more produtive. A major 'target' sector is construction and, more specifically, heavy machinery automation.

Japanese companies  OBAYASHI, NEC, and TAIYU are working together to automate general-purpose construction machinery with a view to dramatically improving productivity while capitalising on their collective experience and technological know-how. As an important first step, the companies have developed an autonomous operation system for backhoes.

The autonomous backhoe system automatically carries out ‘sediment loading’, which makes up a large proportion of the ‘heavy lifting’ in civil engineering - activities such as tunneling or performing underground excavation for large-scale buildings.

But these  tasks demand a high degree of operator skill for obvious reasons - all those arms, booms, and buckets working in real time put  safe operation at a premium.

To get there, NEC applied its Adaptive Predictive Control Technologies to a general-purpose remote control device, the surrogate, which was jointly developed by OBAYASHI and TAIYU, for installation on backhoes. This system combines OBAYASHI's know-how of sediment conditions and dump trucks in order to perform controls that take into account backhoe dynamics and constantly changing conditions. In addition, the use of know-how of technically skilled workers and AI technology has enabled the highly accurate remote control of actions conventionally carried out by technically skilled workers, such as excavation and loading. 

This system is scheduled to be applied at a civil engineering site managed by OBAYASHI in December this year.

The autonomous backhoe operation system sets out to achieve the following:

Accuracy, safety and high productivity

It uses 3D scanners to check the status of embankments in an excavation area. The system determines the points at which the amount of sediment loaded per excavation is maximized, excavates the sediment, swivels to a dump truck standing by, and repeatedly loads the dump truck while monitoring progress by camera. In order to improve the accuracy of operations, the system produces a series of optimal operation plans and utilizes NEC’s Adaptive Predictive Control techniques to understand the dynamic characteristics of the backhoe. In addition, the system analyzes the work data of a large number of skilled technicians to quantify the efficiency of their operations, in addition to analyzing sediment conditions, correct them according to the position of the dump trucks or backhoes, and other conditions in order to realize higher productivity.

When the sediment loaded into a dump truck reaches a specified weight, work stops automatically until another dump truck arrives. Therefore, it is not necessary for personnel to enter a worksite, which helps to ensure safety as well as productivity. In the unlikely event of a worker's entry, Obayashi has installed the "QuattroEyes" system, which was developed by Obayashi, to ensure safety by preventing contact through fail-safe measures.

Easy autonomy compatibility

This system enables backhoes to be operated through a general-purpose remote control, the surrogate, developed jointly by OBAYASHI and TAIYU. The surrogate is attached to the control levers of a backhoe and is compatible with any manufacturer or model. Therefore, the autonomous backhoe operation system can be mounted on any commercially available backhoe, regardless of the type of the system. Since automatic operation and remote control can be easily implemented or removed, it is possible to quickly and flexibly respond to sudden events or complicated operations that require manual control as well.

Integrated control system that saves labour

The autonomous backhoe operation system is managed by a networked control system that integrates and controls a large number of sensors for recognizing the conditions of work areas and construction machinery. Therefore, it is possible for the administrator to remotely check and manage large amounts of information from multiple viewpoints. In the future, 5G next-generation communications will enable faster, larger-capacity, and lower-delay communication, allowing multiple types of construction machinery to operate autonomously at the same time, helping to further improve productivity and labor savings.

Going forward, OBAYASHI will promote the development of next-generation construction production systems capable of realizing high-productivity and high-safety even with few technicians; NEC will leverage the results of this collaboration to develop solutions that can be used to improve the efficiency and safety of construction work; and TAIYU aims to promote the expansion of remote and autonomous operations for a wide range of construction equipment through the training of skilled operators, development of surrogates and other innovative new technologies.