Managing chaos

When too many alarms turn our control rooms chaotic, the problem is often of our own making, but so is the solution. Here are practical tips on launching an alarm management program.


In our daily lives we are sometimes called upon to manage chaotic situations, as anyone with children can attest. The more children one has the more frequent and chaotic the situations are likely to be. Also the more likely that most of the screaming isn’t even real. Soon you treat the chaos like car alarms in California; they’re always going off so everybody ignores them. Of course that means that if someone really is breaking into a car no one tries to stop the thief. I even heard one talking alarm that warned, “Step away from the car,” if you got too close. Of course, that just caused people get close to the car to hear it talk.

Operators’ lives are all about waiting for the chaos to start. Of course that assumes they can tell the real alarms from the warnings, that is. In many of our facilities we have many more things that fall into the warning or even event log categories that flash and sound the horn than we have real alarms, so after a while the operator doesn’t even look at the screen to see if it’s important. They just hit the acknowledge button to silence the horn much like we do with our car alarms.

So if that’s the case why do we bother to create so many alarms? If you understand the history of DCSs, you realize that early in their history the only way to get a record of an event was to alarm it. This was a pretty good advance over older analog systems that typically had no way to print an event. In some industries this led to the use of the first-out annunciators to aid in determining what caused a trip. As control systems progressed so did their capability to host alarms and to create alarm floods because the extra alarm capability didn’t include ways to suppress alarms based on conditions. The best we could do was to assign priorities based on criticality.

But the beast had been unleashed and only recently have there been moves to rethink alarms and how we use them. Unfortunately we have become addicted to the bloody things in ways that I suspect we never anticipated. In the days of light-box annunciators, the cost per alarm and the real estate involved made us think carefully about things to be alarmed. Once alarms effectively became free we used them for everything. Our biggest addiction is to information alarms, those that tell the operator something is going on that someone else thinks he should know about but typically isn’t required to do anything or can’t do anything about. Next in line are alarms caused by an action the operator took such as a low flow alarm because he stopped a pump. During operation the low flow alarm might be a critical alarm but the operator knows he caused a low flow condition so why tell him he did. Yes we had some of those kinds of alarms in the light box days but nothing like the quantity we have now.

For a number of years now we’ve had the tools to change how we use alarms but our biggest hurdles are our addictions. Supervisors still want operators to be aware of what’s going on in their areas by pointing it out to them rather than fixing the display issues that make it hard for an operator to see what’s going on. Projects typically don’t include the time and money, mainly for ROI reasons, to do a proper alarm rationalization. The result is we’re still stuck with those because-I-did-something alarms and alarm floods when a trip occurs. Exacerbating this is the fact that our larger plants are effectively multiple plants with managers that have different operating philosophies driving different implementations of what should be a unified strategy.

So how do we fix this mess we’ve gotten ourselves into?

  • Start by acknowledging the problems it’s causing and recognizing that it can be fixed. Most companies are using some combination of HAZOP, LOPA, and PHA to identify what’s really critical to safety. That can be a starting point for implementing a plant-wide alarm philosophy once one has been developed.
  • Identify the bad actors and target them before anything else. Alarms that chatter because of too little deadband or from being set too close to a normal operating condition on a noisy process or even on a badly tuned loop. Look at normal shutdown periods for alarms that are caused by shutting down.
  • Develop an alarm management strategy and make it someone’s job to ensure that every control system gets reviewed to comply with it. Make sure that this person will have the clout to ensure people participate and the passion to evangelize for it.
  • Understand what tools for implementing the policy are available in your control system. Can it handle conditional alarming? Can it provide alarm help? Can it do alarm shelving? How easy is it to prioritize alarms? How does it capture and present non-alarm events?
  • Piggyback onto projects. Have your champion ensure that any new projects have a budget for implementing the alarm rationalization. This ensures a finite scope to attack rather than just trying to do the whole plant at once.
  • Don’t get discouraged if the first one is a struggle. Even when people have bought into the concept the first one or two projects will be a struggle for people because it will in some cases threaten long held concepts and in others people’s desire for control. If they have not bought in then they may be upset when you question the rationale behind an alarm that they created.
  • Sell the concept that if there is no operator action required, it isn’t an alarm. This will be an ongoing battle with those addicted to informational alarms. An extension of this suggests that even if there is an operator action required, it may still be a low priority alarm based on time to respond and/or potential to do damage.
  • Keep referring back to the criteria you’ve set to make sure that you’re being consistent in your alarm rankings. This is particularly important as you move from project to project or plant area to plant area because as the players change, so does their understanding of what’s written.
  • Don’t be afraid to refine the philosophy. ISA 18.2 is a great starting point for working through the implementation, but you still have to address the specific peculiarities of your plant and your processes.

The good news is you can do this. The bad news is it’s a lot of work and involves a lot of people’s time. So what’s stopping you from taking these steps? Why are you still living with chaos?

This post was written by Bruce Brandt. Bruce is the Technology Leader at MAVERICK Technologies, a leading automation solutions provider offering industrial automation, strategic manufacturing, and enterprise integration services for the process industries. MAVERICK delivers expertise and consulting in a wide variety of areas including industrial automation controls, distributed control systems, manufacturing execution systems, operational strategy, business process optimization and more. 

The Top Plant program honors outstanding manufacturing facilities in North America. View the 2015 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
Pipe fabrication and IIoT; 2017 Product of the Year finalists
The future of electrical safety; Four keys to RPM success; Picking the right weld fume option
A new approach to the Skills Gap; Community colleges may hold the key for manufacturing; 2017 Engineering Leaders Under 40
Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
The cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Power system design for high-performance buildings; mitigating arc flash hazards
VFDs improving motion control applications; Powering automation and IIoT wirelessly; Connecting the dots
Natural gas engines; New applications for fuel cells; Large engines become more efficient; Extending boiler life

Annual Salary Survey

Before the calendar turned, 2016 already had the makings of a pivotal year for manufacturing, and for the world.

There were the big events for the year, including the United States as Partner Country at Hannover Messe in April and the 2016 International Manufacturing Technology Show in Chicago in September. There's also the matter of the U.S. presidential elections in November, which promise to shape policy in manufacturing for years to come.

But the year started with global economic turmoil, as a slowdown in Chinese manufacturing triggered a worldwide stock hiccup that sent values plummeting. The continued plunge in world oil prices has resulted in a slowdown in exploration and, by extension, the manufacture of exploration equipment.

Read more: 2015 Salary Survey

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
The maintenance journey has been a long, slow trek for most manufacturers and has gone from preventive maintenance to predictive maintenance.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This digital report will explore several aspects of how IIoT will transform manufacturing in the coming years.
Maintenance Manager; California Oils Corp.
Associate, Electrical Engineering; Wood Harbinger
Control Systems Engineer; Robert Bosch Corp.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me