Cloud computing, alarm management

Engineering social media: What topics are hot in the LinkedIn Automation & Control Engineering Group? Cloud computing and alarm presentation provoke lively dialogue.

By Jeanine Katzel April 23, 2012

Social media for engineers, such as LinkedIn’s Automation & Control Engineering Group, provide a platform for automation and control engineers to share ideas, opinions, and solutions. CFE Media’s Control Engineering manages and monitors this discussion platform. Below are some of the group’s observations and insights, with more posted online.

Will cloud computing technology be adopted in process control and automation systems? Majed Al Braik, an oil and energy professional in the United Arab Emirates and control engineering section leader at ADMA OPCO, wondered this out loud to fellow members of LinkedIn’s Automaton and Control Engineering Group and drew an array of diverse and interesting insights.

Cloud computing is the way to go, noted Al Braik, for high-level applications of industrial control such as historians, human-machine interfaces, training systems, and asset management. “Adopting this technology at these levels provides beneficial and optimum hardware solutions for offshore applications where space may be limited,” he said, “but security threads may be of concern if the cloud is exposed to a dirty network (for instance, the Internet). However, if the cloud is limited locally, security threads on my process control system may be controllable with appropriate protection and mitigation measures. Cloud computing,” he went on, “is not yet mature enough to be adopted at the controller level, but the time will come when the robust operating and virtualization software systems used in some cloud computing may be available at the process control level.”

Group members agreed and disagreed with Al Braik’s remarks. “It may well happen,” said John Blackburn, design engineer at KAS Paper Systems LTD, Milton Keynes, UK, “but I think it would be a bad move. Too many people are too inventive in ways to put data at risk. You have no idea where critical data are, who is responsible for keeping the data secure, or what the motivation is of those who are keeping the data secure. Proponents of cloud computing may point out all the security technology used to protect a system, but I’ve heard it all before.”

In Blackburn’s view, controls engineers should keep their hardware and software systems under their purview. “Believe me, the investment is worth it,” he said. “You may not know how much so until you’ve had a major security breach, but by then it is too late. Consider the cost of a system failure and malevolent manipulation. That will make the decision for you.”

Blackburn added that he understood the motivation for cloud operation. “Cloud computing is about the virtualization of resources so that they may be shared between users, and about driving down costs through scale and efficiency,” he said. “However, benefits come at a price. Imagine setting up a control system with short-term control loops and fail-safes all close to the system and away from the cloud, taking care to only put higher level, less time-critical functions to the cloud and using only the company’s private cloud. Private encrypted tunnels over public bearers are under your control, but virtualized resources may not be,” he went on, “particularly in a large organization where control of the cloud involves the priorities of many projects, of which yours is just one. The security policies required for your project may be different from other projects where common resource sharing is employed. The cost savings of the cloud, I suspect, will be small compared with the financial consequences of a system compromise.”

Yet there are certainly advantages to implementing cloud strategies, offered Dave Hellyer, vice president, channel sales, Tatsoft LLC, Dixon, Ill., USA, such as lower investment requirements in hardware, software, infrastructure, and personnel to maintain them. “It is not a requirement that all clouds be public,” he said. “Companies looking to modernize their infrastructure and gain these advantages are implementing their own ‘private’ clouds. Another advantage to implementing cloud strategies is that software leveraging cloud technologies provides an engineering environment where multiple engineers can work on the same project from many locations simultaneously, using engineering resources more efficiently and productively.”

Alarm priorities

The discussion on managing alarms was equally plentiful and productive. Initiated by Morgan Wilson, project engineer, Streat Automation, Christchurch, New Zealand, this forum focused on how to find the most effective ways to present alarms—especially critical ones—to plant operators. Alarm lists get out of control quickly, said Wilson. Hundreds can swamp a system. “We reduce them by planning carefully and setting severity levels,” she said, “but ‘acknowledge fever’ can still impact some operators. In addition,” she went on, “pop-up style alarms can be overly intrusive when run on the same SCADA that the operator uses for control. Does anyone have any other ideas for SCADA-based alarming?”

Responding to these queries, Carl Lemp, contract automation engineer, Pfizer, Lincoln, Neb., USA, noted that that Wilson’s post raises several key issues:

  • An overwhelming number of alarms
  • Operators uncertain of what to do when a new alarm occurs
  • “Acknowledge fever,” or confusion resulting from too many alarms to handle at one time. 

“Trying to find a better system to manage alarms after they are generated is like trying to manage spam after it is in your inbox,” said Lemp. “It will always be a struggle.” If possible, he recommended, develop an alarm strategy and decide which alarms can be eliminated. Once unnecessary alarms are removed, those remaining can be streamlined and more easily managed by the alarm management features in the existing system.

Link to more comments from these Automation and Control Engineering Group forums online atop www.controleng.com. LinkedIn members may view the complete discussion or raise questions of their own at LinkedIn.com.

-Jeanine Katzel is a contributing editor to Control Engineering. Contact her at jkatzel@sbcglobal.net. 

Online Extra: Continuing the discussion – What are they talking about? Cloud computing, alarm management stir opinions, suggestions. MORE…

The May 2012 issue of Control Engineering magazine, print and digital edition, featured comments from members of a LinkedIn Automation and Controls group on two popular topics: cloud computing and alarm management. This online extra continues those conversations.

Do you think cloud computing will be adopted in process control and automation systems, asked Majed Al Braik, an oil and energy professional in the United Arab Emirates and control engineering section leader at ADMA OPCO? His query generated comments for and against the use of the popular technology from a number of group members.

John Blackburn, design engineer at KAS Paper Systems LTD, Milton Keynes, UK, cited several concerns about using the cloud, questioning in particular its vulnerability and the inability to control and secure individual projects. Even a secure configuration can deteriorate over time, he said. “A year down the road and a management change later,” he said, “a cloud resourcing company might call and ask why the company is shouldering the burden of cloud computing on its own and offer to do it for the company at significant savings. The new financial director meets with the managing director and the IT director, none of whom are all that interested in your project,” he went on. “They are convinced of the benefits of the move and your cloud resources drift across the globe leaving you with no idea it has even happened until an incident occurs and your phone rings because, after all, the project remains your responsibility.”

Security of systems is a major concern, agreed Mark B. Strube, PE, warehouse control systems analyst, ASICS America, Memphis, Tenn., USA, but system reliability also needs to be considered. “Process control systems running in the cloud are 100% reliant on the facilities network connection to that cloud, aka your Internet connection,” said Strube. “What happens, for instance, if several people check out a video on YouTube? Your Internet connection becomes congested and your network starts running badly, if at all. Best practices for process control networks have always been to keep these systems completely separate from business networks and the Internet for two main reasons: reliability and security. Adding the cloud into the mix would not be wise.”

Peter Lawrence, principal at Inova8, Cleveland, OH, USA, however, is a great believer that the cloud is the future, but only for certain ISA levels. “The security concerns of the cloud have to be balanced with the security concerns of internally hosted applications,” he said. “For instance: How secure is communication which is invariably over a public, not dedicated, channel? And how good is the disaster recovery plan—in practice, not on paper?”

And more on alarms…

Effective alarm system implementation drew equally intense dialogue in response to a request from Morgan Wilson, project engineer, Streat Automation, Christchurch, New Zealand, for information on the most effective ways to present alarms—especially high-importance alarms—to plant operators.

Projects have too many alarms when they are delivered to the plant because engineers think through all the failure modes and then want to alert operators to all possible failures, said Carl Lemp, contract automation engineer, Pfizer, Lincoln, Neb., USA. “Often, the result is multiple alarms for the same failure or alarms for unusual conditions that the operator can’t resolve,” he said. “Add to that a unique alarm strategy for every project, and you have ‘acknowledge fever’ because the alarms don’t have real meaning to the operators.”

Lemp suggested controls engineers take a step or two back and decide which alarms can be eliminated. “The plant needs to develop an alarm strategy,” he said. “What does an alarm mean? Is it to get the operator’s attention, to prompt action, or both? If it is both, then the alarms should be set so that the operator can easily tell the difference. What does it mean for an operator to acknowledge an alarm?” he continued. “Does acknowledge mean the operator takes responsibility for doing something about the alarm, or just that he heard it? Does a critical alarm mean equipment, product, or personnel are in danger, or does it just mean that some engineer thought it was important? Once unnecessary alarms are eliminated, remaining alarms can be streamlined by setting appropriate limits and by eliminating chatter and duplicates. Remaining alarms can be managed more easily by the alarm management features built into the existing system.”

Dannie Hughes, automation engineer at SSAB, Mobile, Ala., USA, had similar issues with the alarm system used at his plant, which had more than 6,000 alarms with possibly dozens active at any one time. “Operators regularly missed new alarms due to nuisance alarms,” said Hughes. “We tried pop-up alarms, but they sometimes covered up critical HMI information at inopportune times. We eventually found an audible voice alarm to be most effective. It connects to our system using OPC and announces or alarms over two-way radios using text-to-speech. The operators are informed of the alarms without being distracted from the process. In addition,” Hughes went on, “our maintenance people are informed about the alarm condition even though they are not in front of an HMI. They can react and resolve the situation faster.”

Properly setting alarm priorities greatly helps the operator focus attention on the most important alarms, added Oleg Rojkovski, PEng, control systems engineer, Spartan Controls, Calgary, AB, Canada. “I’ve been through a few alarm rationalization exercises and sometimes the resulting recommendations contradict what the operators are used to,” he said. “Consequential alarms are always a problem. I don’t know of an easy way of dealing with them, other than writing logic which would conditionally suppress/unsuppress them. It is time and resource consuming and requires a great deal of understanding of the process. It is also important to maintain the alarm system properly,” he continued. “Plants and process conditions change over time, requiring almost continuous adjustments to the alarm system. There are good tools available to help in alarm system analysis and benchmarking, but implementing conditional alarming typically still requires a programmer.”

Bruce Brandt, principal engineer, Maverick Technologies, Houston, Texas, USA, refers control engineers to International Society of Automation (ISA) standard 18.2 [Management of Alarm Systems for the Process Industries; www.isa.org] and the Abnormal Situation Management consortium [www.asmconsortium.net] website. “I highly recommend both,” said Brandt. “The most important take-away is the definition of an alarm. To be defined as an alarm, the event must require the operator take an action within a specified time to prevent or lessen the consequence of the alarm—period. Anything else is an event the operator may want to know has occurred through a notification on screen or in an event log.”

Next, use the guidelines in ISA 18.2 to develop your plant’s alarm philosophy, advised Brandt. “This includes developing a matrix that considers the consequences of events you are proposing to alarm with respect to its impact on health and safety, environmental impact, and financial impact,” he said. “Since these areas are usually linked, the more severe the event, the greater the impact on all three. They should be placed across the top of the matrix. The time to respond goes down the left side. The higher the severity is, or the shorter the available time to respond, the higher the priority of the alarm. The highest priority is the event with the most impact and the least amount of time to respond.”

Finally, Brandt suggested the company engage a trained resource to take the plant staff through a rationalization of alarms. “In the rationalization process,” he said, “all existing alarms—or, for new sites, proposed alarms—are reviewed and questions asked: What causes this event to happen? What are the consequences of it happening? What action must the operator take? How long does the operator have to act? How severe (according to the matrix) is the event? The answers to these questions will reveal if you have ‘an alarm’ and its priority. For a rationalization to be performed effectively, operators, process personnel, and process control personnel should be included in the sessions. The plant should also develop a management of change process to maintain alarms. In the end, operators should have no more than one alarm every 5 to 10 minutes and no floods on a trip.”

Based on this discussion, not enough people appear to be implementing systems that give operators more information about alarms as they come up, concluded Wilson. “Are there automation techniques that can be used to ensure operators take action on the alarms, instead of simply hitting the acknowledge button?” she mused. Having operators who really know the process is key, she added, “but what happens when that person is away or isn’t sure exactly what to do about the alarm? Of course having a reasonably stable process and correctly configured alarms will mean the operator is not dealing with ‘nuisance’ alarms. Having a robust alarm system doesn’t mean eliminating all alarms. They are there for good reason and should be presented in the most efficient and user friendly way possible.”

Read the original print article above. LinkedIn members may view the complete discussion or raise questions of their own at www.LinkedIn.com. 

U.S. Manufacturing: Engineering Social Media – Is Industry Returning to the U.S.? https://bit.ly/wzNbvG 

Social media for engineers: LinkedIn Automation & Controls group

https://www.linkedin.com/groups/Automation-Control-Engineering-1967039?gid=1967039 

Has social media combined with process control systems yet? Were you fooled?  https://bit.ly/HPAl6I