Gestures can control cranes

Haptic control is proven for cranes, by using a Microsoft Kinect system.


A loader crane operator using a Microsoft Kinect sensor can control loader crane movements, programming motion with gestures, voice, and recognition algorithms. Matlab/Simulink and C++ programming help. Courtesy: West Pomeranian University of Technology,Loader cranes play a significant role in the industry, but despite seemingly simple construction, they are very difficult to control. Advanced wireless controls may give way to the use of operator hand gestures to control the cranes’ motion. Cranes are used all over the world at building sites, on ships, at transportation depots, at ports, and wherever heavy cargo needs to be lifted. They are also used by emergency services and military personnel (for disarming mines, for example).

To control loader cranes, companies employ skilled operators, rigorously trained to increase the efficiency of their work and to minimize the risk of accidents. One of the factors influencing loader crane control is a natural tendency to oscillate loads. Because of oscillations, it is a challenge even for experienced operators to quickly, safely, and accurately move a load without floating it. Too rapid movements of the boom could lead to load swing, and thus destroy the crane, damaging its mechanisms or causing a roll-over of the vehicle on which the crane is mounted. The next challenge for the loader crane operator is to work with hydraulic levers to lift cargo, which can be located out of sight (behind the wall or much higher). In this case, it is practically impossible for an individual operator to precisely move the cargo to a designated place. Doing this requires moving away from the control interface to check the load or ask for help from other people, which in turn reduces safety and productivity.

Crane control

In recent years, despite new technology, construction cranes, control algorithms, and methods for damping load oscillations, little attention has been paid to crane control interfaces and their influence on operator efficiency. The control method of most cranes has not changed for years. In most cases, levers are connected directly to the hydraulic system, control panels, and joysticks. These interfaces receive operator commands, such as pressing a button, which sends a signal to the computer, where it is processed using a suitable algorithm. Then a control signal is sent to the engine, for example, to rotate the crane base.

A recent market trend is the use of radio remote control panels to control loading cranes. Among equipment loader cranes, spider cranes, and forest cranes, remote control panels increasingly are being offered as a standard feature, not an option. Remote control panels are the most advanced control equipment designed for crane handling, equipped with joysticks, electronic levers, buttons, switches, and LED and LCD displays. These elements allow for more precise control, perform more accurate moves, and set many crane parameters without the operator having to enter the cabin or go to the console. Currently, manufacturers are working on increasing the amount of data exchanged between the remote controllers and the crane system, which are shown on the LCD display. In the latest panels, functions of the lever or joystick are shown on the LCD display and change automatically depending on the mode.

Remote control

Control loader cranes with a radio remote control panel are more effective and safer. This method allows one operator to attach a load to the hook and move it to the designated place. This is possible because the operator is free to move with the wireless control panel and can send a command to the crane at any time. While hitching a load, the operator can approach it and then go to a place where the cargo will be moved. In this way, the operator always has a clear view of the load and the environment, minimizing the risk of error or accident. The "Fassi SmartApp" is an Android OS that allows a user to read status information and control the loader crane with a smartphone. As of spring 2012, it was still in the testing phase.

Previously described methods of control by loading cranes are very intuitive and based primarily on the use of levers and joysticks. In addition, they require the operator to learn particular movements and to remember large amounts of information (identification LEDs, switches, shares caused by the deflection of the joystick), and they do not reflect the movement of the load.

Hand gesture control cranes

Loader cranes can be controlled by an interface based on the operator’s gestures and tracking movements. Proof of concept uses the Microsoft Kinect motion controller for tracking the operator. The Microsoft Kinect motion controller receives the spatial positions of 20 points on the body. The data is obtained for control and gesture recognition algorithms.

The system was implemented using the Matlab/Simulink environment, using the "Natural Interaction Device" package, Kinect SDK and Developer Toolkit, and C++ language. Two methods to control the loader crane have been devised. The first uses the Microsoft Kinect sensor to track the position of the operator's hand. The algorithm using inverse kinematics processes the obtained data with hand location. This allows the operator to control the end of the jib crane directly with hand movements. It is a natural and intuitive way to control, because it imitates the daily action of moving objects.

The second control method uses hand tracking and gesture recognition. Gestures signal various crane functions, for example, ejecting mainstays, controlling individual cylinders, or returning the crane to the transport position. Future plans include further system development with new gestures, the location of the crane operator, and the addition of voice commands.

As shown, the growth in computing power, virtual reality, 3D environments, and the use of gestures and human senses to control all types of devices (haptic feedback) can be used to change the way cranes are controlled, or create virtual environments for remote control of loader cranes. The manufacturers of these machines are increasingly recognizing that virtual technologies can move beyond the world of entertainment to play an important role in industry.

- Krzysztof Pietrusewicz and Karol Miądlicki are from West Pomeranian University of Technology, Szczecin, and contributors for Control Engineering Poland. Edited by Mark T. Hoske, content manager, CFE Media, Control Engineering,


Below see links to other related articles.

No comments
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2015 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
2015 Top Plant: Phoenix Contact, Middletown, Pa.; 2015 Best Practices: Automation, Electrical Safety, Electrical Systems, Pneumatics, Material Handling, Mechanical Systems
A cool solution: Collaboration, chemistry leads to foundry coat product development; See the 2015 Product of the Year Finalists
Raising the standard: What's new with NFPA 70E; A global view of manufacturing; Maintenance data; Fit bearings properly
Special report: U.S. natural gas; LNG transport technologies evolve to meet market demand; Understanding new methane regulations; Predictive maintenance for gas pipeline compressors
Cyber security cost-efficient for industrial control systems; Extracting full value from operational data; Managing cyber security risks
Drilling for Big Data: Managing the flow of information; Big data drilldown series: Challenge and opportunity; OT to IT: Creating a circle of improvement; Industry loses best workers, again
Migrating industrial networks; Tracking HMI advances; Making the right automation changes
Understanding transfer switch operation; Coordinating protective devices; Analyzing NEC 2014 changes; Cooling data centers
Upgrading secondary control systems; Keeping enclosures conditioned; Diagnostics increase equipment uptime; Mechatronics simplifies machine design

Annual Salary Survey

After almost a decade of uncertainty, the confidence of plant floor managers is soaring. Even with a number of challenges and while implementing new technologies, there is a renewed sense of optimism among plant managers about their business and their future.

The respondents to the 2014 Plant Engineering Salary Survey come from throughout the U.S. and serve a variety of industries, but they are uniform in their optimism about manufacturing. This year’s survey found 79% consider manufacturing a secure career. That’s up from 75% in 2013 and significantly higher than the 63% figure when Plant Engineering first started asking that question a decade ago.

Read more: 2014 Salary Survey: Confidence rises amid the challenges

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
This article collection contains several articles on the vital role that compressed air plays in manufacturing plants.