A team approach to resolving process upsets

Maintain business and process reliability with 24/7 knowledge management

By Dr. Joel Shertok and Andreas Eschbach February 8, 2021

Process upsets occur, and they need to be anticipated and controlled. Unfortunately, upsets are endemic to plant operations within the chemical industry, despite the advanced technology intended to keep operations smooth and stable.

An upset is the result of one or more process-related issues that disrupt the intended operations of a chemical process. An upset can be a minor issue like an incremental temperature increase over time, which can lead to an out-of-spec formulation. It also could be a major upset, such as an out of control reaction, which could compromise the environment and worker safety within the entire plant if not quickly identified and corrected.

Whether an upset has a major impact or not, managing upsets is time consuming and creates diversions for plant management and operations. This can significantly slow down production schedules. What can make upsets even more significant in the chemical industry is the amount of time it takes to correct them: Investigations and resolution can take hours, across more than one shift.

Production teams often lack the digital tools needed to access information immediately through knowledge capture and autonomous communications during shift handovers. This causes substantial wasted time searching for information, generating status updates, investigating results and reporting across siloed operations and multiple shifts.

If the shift relief team cannot access all findings, they may have to redo the investigations and operations. Instead of a straightforward resolution of the upset, plant management may be forced to double back and repeat previous efforts.

Missing key information is not an option. Every hour an upset is allowed to persist unresolved can impact profits and potentially create catastrophic safety hazards. In fact, an operating unit’s response to an upset can be critical to the unit’s process reliability. The success and reliability of a unit are measured by how well the organization responds to the upset.

Communication challenges

There can be several teams operating over different shifts with multiple staff involved performing a variety of functions, depending on the size of the operation. An operation can run 24×7 and thus can be unforgiving of process issues. A single process plant can consist of more than 60 different personnel and associated functions. It is imperative that these groups have the opportunity to communicate to gain a common understanding of what is known and unknown, to effectively collaborate in solving a problem (see Figure 1).

When plant personnel have to send multiple emails while maintaining multiple spreadsheets and documents simultaneously during an upset, the various functions will have increasing difficulties in getting an overview of the essential issues. Therefore, plant functions can be challenged with multiple sources of information. In addition, they are more likely to fall into the trap of mistaking assumptions for facts. For any upset, the real facts must be discerned and confirmed to ensure the proper response. Shift changes, with new personnel coming on to the scene, make the situation even more challenging.

Remember, a symptom is not the problem but rather points to the problem. A collection of symptoms fully researched, can identify the underlying problem, much like a physician making a diagnosis.

Creating reliability

Plant upsets are continuous in nature and are not always resolved by the end of a shift. Therefore, managing both within a given shift and over shift changes in a 24/7 operation needs a digital solution. The operating team must have a system in place for updating the relief shifts on progress and critical activities.

Digital technologies, like plant process management (PPM), provide a single platform to bring a “single truth” to all involved and enable production teams to address process upsets more effectively (see Figure 2). This makes operations more reliable, dependable and safer. Achieving continuous stability in the face of consistent process challenges is a measurement of this reliability.

Providing a single communications platform to all the impacted functions significantly enhances the timely resolution of process upsets. This ensures that the resolution activities go hand in hand across shift changes. Inter- and intra-plant communications must be seamless to avoid time consuming repeat investigations and attempted solutions that are deficient.

The chemical industry lacks digitalization

A recent survey by 451 Research indicated that chemical operators are ill equipped for digital knowledge transfer. Regarding PPM solutions, research shows that plants in the chemical industry are, for the most part, digitally unprepared to assure smooth internal/external communications. Only 10% of 300 senior executives responded to a recent survey that their companies are currently in the execution stage of a digital strategy. The majority of these companies are still collecting data via manual means (or not at all).

Despite these findings, there is still a high degree of interest in digitalization. Safety and process optimization are drivers of corporate goals. The research concluded:

  • Inter-shift and inter-functional communications are typically carried out verbally
  • Offsite personnel are unfortunately often not directly informed; they are usually brought up to speed by “back-channels”
  • Maintenance and engineering work are recorded on standard forms, which are then filed away and not readily accessible
  • Paper reports are issued, but likely also filed away and eventually forgotten.
  • Reliability can be greatly enhanced by establishing an “improvement cycle.” The following four steps support ongoing reliability:
  • Identification of the problem (follow the symptoms)
  • Communication (PPM software platform assuring shift collaboration)
  • Corrective action (document the steps taken to solve the upset)
  • Retrospective documentation (written, not verbal, communications are archived to the knowledge base).

Step four is often glossed over. The retrospective documentation should outline and validate all the actions taken. People often tend to skip this step; the problem is solved and now it’s business as usual. But, by doing this validation step, in writing, and adding it to a PPM platform, a knowledge base will be built that could be used to solve a similar issue or serve as a repository, accessible to all employees.

Creating a knowledge base

COVID-19 has upended the normal way of managing processes. With remote working situations, senior management must be able to contribute from remote locations. This significantly increases the need for a digital system. Staff can no longer rely on physical access to documents and personnel. Paper documentation, like spreadsheets, can create access issues with so many personnel now residing off-site.

Digital access simplifies handover tasks from remote engineers to shift teams. No longer does it matter where a particular functional person resides — at the plant, at home or at a remote site. Companies need to invest in software platforms that capture shift handover information. Today these platforms are designed to ensure transparency, reliability and visibility across all plant functions and roles — helping teams become well integrated.

It now becomes possible to integrate PPM platforms with other plant mission-critical systems. These include such systems as computerized maintenance management systems (CMMS) like SAP PM or IBM Maximo; and process historian programs such as OSIsoft PI or LIMS systems.

This unification enables process digitalization, and allows evolution to a more reliable, knowledge-based organization that can quickly and successfully overcome process upsets. This process also results in a significant financial benefit for the company involved. For example, using a commodity cumene/phenol plant as a base case, an improvement in their process upset response could easily result in a significant annual savings. A one-day shutdown, or the production of unsalable product, can lead to a loss of $112,000 or $56,000/shift. This is a significant expense, assuming no associated equipment damage is found during inspections.

Implementation of modern, state of the art digital technology can lead to significant improvements in both plant process safety associated plant economics.


Author Bio: Dr. Joel Shertok is Chemical Engineering and Materials Consultant at eschbach. Andreas Eschbach is the CEO of eschbach.