How can robots get our attention?

Research helps robots function more in tune with human coworkers by picking up on nonverbal cues: Georgia Institute of Technology. Smarter software and machine vision could make industrial robots safer and more efficient.


Getting someone’sGeorgia Institute of Technology - Simon attention can be easy with a loud noise or a shout, but what if the situation calls for a little more tact? How can a robot use subtle cues to attract a human’s notice and tell when it has captured it? In a preliminary study, researchers at the Georgia Institute of Technology have found that they can program a robot to understand when it gains a human’s attention and when it falls short. The research is being presented at the Human-Robot Interaction conference in Lausanne, Switzerland.

Control Engineering context: Such research could be helpful for industrial robots, which could use such software to change their behavior or routes depending on interactions with human counterparts in the workplace. Automated guided vehicles (AGVs), for instance, might eventually slow slightly less, or take a more direct route in a situation, if it has a cue that a human nearby sees where the roving robot is at and where it’s going. Such software could add productivity to the industrial applications without increasing risk. Software that helps computers better understand human language also advanced. For more about AGVs from Control Engineering, see the AIMing for Automated Vehicles blog at

“The primary focus was trying to give Simon, our robot, the ability to understand when a human being seems to be reacting appropriately, or in some sense is interested now in a response with respect to Simon and to be able to do it using a visual medium, a camera,” said Aaron Bobick, professor and chair of the School Georgia Institute of Technology - Simon Visionof Interactive Computing in Georgia Tech’s College of Computing.

Using the socially expressive robot Simon, from Assistant Professor Andrea Thomaz’s Socially Intelligent Machines lab, researchers wanted to see if they could tell when he had successfully attracted the attention of a human who was busily engaged in a task and when he had not.

“Simon would make some form of a gesture, or some form of an action when the user was present, and the computer vision task was to try to determine whether or not you had captured the attention of the human being,” said Bobick.

With close to 80 percent accuracy Simon was able to tell, using only his cameras as a guide, whether someone was paying attention to him or ignoring him.

“We would like to bring robots into the human world. That means they have to engage with human beings, and human beings have an expectation of being engaged in a way similar to the way other human beings would engage with them,” said Bobick.

“Other human beings understand turn-taking. They understand that if I make some indication, they’ll turn and face someone when they want to engage with them and they won’t when they don’t want to engage with them. In order for these robots to work with us effectively, they have to obey these same kinds of social conventions, which means they have to perceive the same thing humans perceive in determining how to abide by those conventions,” he added.

Researchers plan to go further with their investigations into how Simon can read communication cues by studying whether he can tell by a person’s gaze whether they are paying attention or using elements of language or other actions.

“Previously people would have pre-defined notions of what the user should do in a particular context and they would look for those,” said Bobick. “That only works when the person behaves exactly as expected. Our approach, which I think is the most novel element, is to use the user’s current behavior as the baseline and observe what changes.”

The research team for this study consisted of Bobick, Thomaz, doctoral student Jinhan Lee and undergraduate student Jeffrey Kiser.

- Edited by Gust Gianos, Control Engineering,

No comments
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2013 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
A cool solution: Collaboration, chemistry leads to foundry coat product development; See the 2015 Product of the Year Finalists
Raising the standard: What's new with NFPA 70E; A global view of manufacturing; Maintenance data; Fit bearings properly
Sister act: Building on their father's legacy, a new generation moves Bales Metal Surface Solutions forward; Meet the 2015 Engineering Leaders Under 40
Cyber security cost-efficient for industrial control systems; Extracting full value from operational data; Managing cyber security risks
Drilling for Big Data: Managing the flow of information; Big data drilldown series: Challenge and opportunity; OT to IT: Creating a circle of improvement; Industry loses best workers, again
Pipeline vulnerabilities? Securing hydrocarbon transit; Predictive analytics hit the mainstream; Dirty pipelines decrease flow, production—pig your line; Ensuring pipeline physical and cyber security
Upgrading secondary control systems; Keeping enclosures conditioned; Diagnostics increase equipment uptime; Mechatronics simplifies machine design
Designing positive-energy buildings; Ensuring power quality; Complying with NFPA 110; Minimizing arc flash hazards
Building high availability into industrial computers; Of key metrics and myth busting; The truth about five common VFD myths

Annual Salary Survey

After almost a decade of uncertainty, the confidence of plant floor managers is soaring. Even with a number of challenges and while implementing new technologies, there is a renewed sense of optimism among plant managers about their business and their future.

The respondents to the 2014 Plant Engineering Salary Survey come from throughout the U.S. and serve a variety of industries, but they are uniform in their optimism about manufacturing. This year’s survey found 79% consider manufacturing a secure career. That’s up from 75% in 2013 and significantly higher than the 63% figure when Plant Engineering first started asking that question a decade ago.

Read more: 2014 Salary Survey: Confidence rises amid the challenges

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.