Gestures can control cranes

Haptic control is proven for cranes, by using a Microsoft Kinect system.


A loader crane operator using a Microsoft Kinect sensor can control loader crane movements, programming motion with gestures, voice, and recognition algorithms. Matlab/Simulink and C++ programming help. Courtesy: West Pomeranian University of Technology,Loader cranes play a significant role in the industry, but despite seemingly simple construction, they are very difficult to control. Advanced wireless controls may give way to the use of operator hand gestures to control the cranes’ motion. Cranes are used all over the world at building sites, on ships, at transportation depots, at ports, and wherever heavy cargo needs to be lifted. They are also used by emergency services and military personnel (for disarming mines, for example).

To control loader cranes, companies employ skilled operators, rigorously trained to increase the efficiency of their work and to minimize the risk of accidents. One of the factors influencing loader crane control is a natural tendency to oscillate loads. Because of oscillations, it is a challenge even for experienced operators to quickly, safely, and accurately move a load without floating it. Too rapid movements of the boom could lead to load swing, and thus destroy the crane, damaging its mechanisms or causing a roll-over of the vehicle on which the crane is mounted. The next challenge for the loader crane operator is to work with hydraulic levers to lift cargo, which can be located out of sight (behind the wall or much higher). In this case, it is practically impossible for an individual operator to precisely move the cargo to a designated place. Doing this requires moving away from the control interface to check the load or ask for help from other people, which in turn reduces safety and productivity.

Crane control

In recent years, despite new technology, construction cranes, control algorithms, and methods for damping load oscillations, little attention has been paid to crane control interfaces and their influence on operator efficiency. The control method of most cranes has not changed for years. In most cases, levers are connected directly to the hydraulic system, control panels, and joysticks. These interfaces receive operator commands, such as pressing a button, which sends a signal to the computer, where it is processed using a suitable algorithm. Then a control signal is sent to the engine, for example, to rotate the crane base.

A recent market trend is the use of radio remote control panels to control loading cranes. Among equipment loader cranes, spider cranes, and forest cranes, remote control panels increasingly are being offered as a standard feature, not an option. Remote control panels are the most advanced control equipment designed for crane handling, equipped with joysticks, electronic levers, buttons, switches, and LED and LCD displays. These elements allow for more precise control, perform more accurate moves, and set many crane parameters without the operator having to enter the cabin or go to the console. Currently, manufacturers are working on increasing the amount of data exchanged between the remote controllers and the crane system, which are shown on the LCD display. In the latest panels, functions of the lever or joystick are shown on the LCD display and change automatically depending on the mode.

Remote control

Control loader cranes with a radio remote control panel are more effective and safer. This method allows one operator to attach a load to the hook and move it to the designated place. This is possible because the operator is free to move with the wireless control panel and can send a command to the crane at any time. While hitching a load, the operator can approach it and then go to a place where the cargo will be moved. In this way, the operator always has a clear view of the load and the environment, minimizing the risk of error or accident. The "Fassi SmartApp" is an Android OS that allows a user to read status information and control the loader crane with a smartphone. As of spring 2012, it was still in the testing phase.

Previously described methods of control by loading cranes are very intuitive and based primarily on the use of levers and joysticks. In addition, they require the operator to learn particular movements and to remember large amounts of information (identification LEDs, switches, shares caused by the deflection of the joystick), and they do not reflect the movement of the load.

Hand gesture control cranes

Loader cranes can be controlled by an interface based on the operator’s gestures and tracking movements. Proof of concept uses the Microsoft Kinect motion controller for tracking the operator. The Microsoft Kinect motion controller receives the spatial positions of 20 points on the body. The data is obtained for control and gesture recognition algorithms.

The system was implemented using the Matlab/Simulink environment, using the "Natural Interaction Device" package, Kinect SDK and Developer Toolkit, and C++ language. Two methods to control the loader crane have been devised. The first uses the Microsoft Kinect sensor to track the position of the operator's hand. The algorithm using inverse kinematics processes the obtained data with hand location. This allows the operator to control the end of the jib crane directly with hand movements. It is a natural and intuitive way to control, because it imitates the daily action of moving objects.

The second control method uses hand tracking and gesture recognition. Gestures signal various crane functions, for example, ejecting mainstays, controlling individual cylinders, or returning the crane to the transport position. Future plans include further system development with new gestures, the location of the crane operator, and the addition of voice commands.

As shown, the growth in computing power, virtual reality, 3D environments, and the use of gestures and human senses to control all types of devices (haptic feedback) can be used to change the way cranes are controlled, or create virtual environments for remote control of loader cranes. The manufacturers of these machines are increasingly recognizing that virtual technologies can move beyond the world of entertainment to play an important role in industry.

- Krzysztof Pietrusewicz and Karol Miądlicki are from West Pomeranian University of Technology, Szczecin, and contributors for Control Engineering Poland. Edited by Mark T. Hoske, content manager, CFE Media, Control Engineering, mhoske(at)


Below see links to other related articles.

No comments
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2013 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
The true cost of lubrication: Three keys to consider when evaluating oils; Plant Engineering Lubrication Guide; 11 ways to protect bearing assets; Is lubrication part of your KPIs?
Contract maintenance: 5 ways to keep things humming while keeping an eye on costs; Pneumatic systems; Energy monitoring; The sixth 'S' is safety
Transport your data: Supply chain information critical to operational excellence; High-voltage faults; Portable cooling; Safety automation isn't automatic
Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Plant Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.

Maintaining low data center PUE; Using eco mode in UPS systems; Commissioning electrical and power systems; Exploring dc power distribution alternatives
Synchronizing industrial Ethernet networks; Selecting protocol conversion gateways; Integrating HMIs with PLCs and PACs
Why manufacturers need to see energy in a different light: Current approaches to energy management yield quick savings, but leave plant managers searching for ways of improving on those early gains.

Annual Salary Survey

Participate in the 2013 Salary Survey

In a year when manufacturing continued to lead the economic rebound, it makes sense that plant manager bonuses rebounded. Plant Engineering’s annual Salary Survey shows both wages and bonuses rose in 2012 after a retreat the year before.

Average salary across all job titles for plant floor management rose 3.5% to $95,446, and bonus compensation jumped to $15,162, a 4.2% increase from the 2010 level and double the 2011 total, which showed a sharp drop in bonus.

2012 Salary Survey Analysis

2012 Salary Survey Results

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.