Gestures can control cranes

Haptic control is proven for cranes, by using a Microsoft Kinect system.


A loader crane operator using a Microsoft Kinect sensor can control loader crane movements, programming motion with gestures, voice, and recognition algorithms. Matlab/Simulink and C++ programming help. Courtesy: West Pomeranian University of Technology,Loader cranes play a significant role in the industry, but despite seemingly simple construction, they are very difficult to control. Advanced wireless controls may give way to the use of operator hand gestures to control the cranes’ motion. Cranes are used all over the world at building sites, on ships, at transportation depots, at ports, and wherever heavy cargo needs to be lifted. They are also used by emergency services and military personnel (for disarming mines, for example).

To control loader cranes, companies employ skilled operators, rigorously trained to increase the efficiency of their work and to minimize the risk of accidents. One of the factors influencing loader crane control is a natural tendency to oscillate loads. Because of oscillations, it is a challenge even for experienced operators to quickly, safely, and accurately move a load without floating it. Too rapid movements of the boom could lead to load swing, and thus destroy the crane, damaging its mechanisms or causing a roll-over of the vehicle on which the crane is mounted. The next challenge for the loader crane operator is to work with hydraulic levers to lift cargo, which can be located out of sight (behind the wall or much higher). In this case, it is practically impossible for an individual operator to precisely move the cargo to a designated place. Doing this requires moving away from the control interface to check the load or ask for help from other people, which in turn reduces safety and productivity.

Crane control

In recent years, despite new technology, construction cranes, control algorithms, and methods for damping load oscillations, little attention has been paid to crane control interfaces and their influence on operator efficiency. The control method of most cranes has not changed for years. In most cases, levers are connected directly to the hydraulic system, control panels, and joysticks. These interfaces receive operator commands, such as pressing a button, which sends a signal to the computer, where it is processed using a suitable algorithm. Then a control signal is sent to the engine, for example, to rotate the crane base.

A recent market trend is the use of radio remote control panels to control loading cranes. Among equipment loader cranes, spider cranes, and forest cranes, remote control panels increasingly are being offered as a standard feature, not an option. Remote control panels are the most advanced control equipment designed for crane handling, equipped with joysticks, electronic levers, buttons, switches, and LED and LCD displays. These elements allow for more precise control, perform more accurate moves, and set many crane parameters without the operator having to enter the cabin or go to the console. Currently, manufacturers are working on increasing the amount of data exchanged between the remote controllers and the crane system, which are shown on the LCD display. In the latest panels, functions of the lever or joystick are shown on the LCD display and change automatically depending on the mode.

Remote control

Control loader cranes with a radio remote control panel are more effective and safer. This method allows one operator to attach a load to the hook and move it to the designated place. This is possible because the operator is free to move with the wireless control panel and can send a command to the crane at any time. While hitching a load, the operator can approach it and then go to a place where the cargo will be moved. In this way, the operator always has a clear view of the load and the environment, minimizing the risk of error or accident. The "Fassi SmartApp" is an Android OS that allows a user to read status information and control the loader crane with a smartphone. As of spring 2012, it was still in the testing phase.

Previously described methods of control by loading cranes are very intuitive and based primarily on the use of levers and joysticks. In addition, they require the operator to learn particular movements and to remember large amounts of information (identification LEDs, switches, shares caused by the deflection of the joystick), and they do not reflect the movement of the load.

Hand gesture control cranes

Loader cranes can be controlled by an interface based on the operator’s gestures and tracking movements. Proof of concept uses the Microsoft Kinect motion controller for tracking the operator. The Microsoft Kinect motion controller receives the spatial positions of 20 points on the body. The data is obtained for control and gesture recognition algorithms.

The system was implemented using the Matlab/Simulink environment, using the "Natural Interaction Device" package, Kinect SDK and Developer Toolkit, and C++ language. Two methods to control the loader crane have been devised. The first uses the Microsoft Kinect sensor to track the position of the operator's hand. The algorithm using inverse kinematics processes the obtained data with hand location. This allows the operator to control the end of the jib crane directly with hand movements. It is a natural and intuitive way to control, because it imitates the daily action of moving objects.

The second control method uses hand tracking and gesture recognition. Gestures signal various crane functions, for example, ejecting mainstays, controlling individual cylinders, or returning the crane to the transport position. Future plans include further system development with new gestures, the location of the crane operator, and the addition of voice commands.

As shown, the growth in computing power, virtual reality, 3D environments, and the use of gestures and human senses to control all types of devices (haptic feedback) can be used to change the way cranes are controlled, or create virtual environments for remote control of loader cranes. The manufacturers of these machines are increasingly recognizing that virtual technologies can move beyond the world of entertainment to play an important role in industry.

- Krzysztof Pietrusewicz and Karol Miądlicki are from West Pomeranian University of Technology, Szczecin, and contributors for Control Engineering Poland. Edited by Mark T. Hoske, content manager, CFE Media, Control Engineering,


Below see links to other related articles.

The Top Plant program honors outstanding manufacturing facilities in North America. View the 2015 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
Doubling down on digital manufacturing; Data driving predictive maintenance; Electric motors and generators; Rewarding operational improvement
2017 Lubrication Guide; Software tools; Microgrids and energy strategies; Use robots effectively
Prescriptive maintenance; Hannover Messe 2017 recap; Reduce welding errors
The cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Mobility as the means to offshore innovation; Preventing another Deepwater Horizon; ROVs as subsea robots; SCADA and the radio spectrum
Research team developing Tesla coil designs; Implementing wireless process sensing
Commissioning electrical systems; Designing emergency and standby generator systems; Paralleling switchgear generator systems
Natural gas engines; New applications for fuel cells; Large engines become more efficient; Extending boiler life

Annual Salary Survey

Before the calendar turned, 2016 already had the makings of a pivotal year for manufacturing, and for the world.

There were the big events for the year, including the United States as Partner Country at Hannover Messe in April and the 2016 International Manufacturing Technology Show in Chicago in September. There's also the matter of the U.S. presidential elections in November, which promise to shape policy in manufacturing for years to come.

But the year started with global economic turmoil, as a slowdown in Chinese manufacturing triggered a worldwide stock hiccup that sent values plummeting. The continued plunge in world oil prices has resulted in a slowdown in exploration and, by extension, the manufacture of exploration equipment.

Read more: 2015 Salary Survey

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
The maintenance journey has been a long, slow trek for most manufacturers and has gone from preventive maintenance to predictive maintenance.
Featured articles highlight technologies that enable the Industrial Internet of Things, IIoT-related products and strategies to get data more easily to the user.
This digital report will explore several aspects of how IIoT will transform manufacturing in the coming years.
Maintenance Manager; California Oils Corp.
Associate, Electrical Engineering; Wood Harbinger
Control Systems Engineer; Robert Bosch Corp.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me