Do not ignore those recurring errors in manufacturing or processing

Error messages matter, a lot. Find them, track them, and remove them. Adopt a 4-point no-errors policy. Can you answer 4 questions about errors, warnings, and alarms in manufacturing and processing? Learn from Anthony Baker.

07/26/2013


You know the manufacturing automation or process control errors we are talking about: The nagging errors in the log that don’t seem to go away. The system is “running fine” but there’s a tag in the PLC or other controller that doesn’t exist, or a link in the application that doesn’t compile completely. The lesson here is simple, but one that lots of engineers ignore:

NO ERRORS NO WARNINGS.

No errors, no warnings, advises Callisto IntegrationAnthony Baker received a recent call from a very stressed out, very important client. The client had an human-machine interface (HMI) software system that would become unresponsive after 12-15 hours of uptime. The onsite engineers were unsure of the cause and had called Anthony as a last resort. The system was in a fragile state, and production was at risk.

This was a new site for Anthony and, despite having a deep understanding of this particular application, Anthony did what lots of engineers do and went into the application error logs, created a list of errors and one-by-one hunted down the root cause and eliminated the message. In many cases the error was a warning such as “Tagname ZZ does not exist in the PLC” or “Database connection Y has timed out.” All the errors could have been prevented with clean implementation and more thorough test plans that include a “NO ERRORS NO WARNINGS” policy.

Eliminating the errors in the logs one by one, the issue was eventually solved. As a note, the most frequent and obvious error was actually not the root cause of the outage — it was a subtle, one-time issue that cropped up in the logs and was dutifully logged by the HMI application. Unfortunately, the root-cause error was drowned out by a recurring issue that had been present in the system for years. Only by fixing this error flooding the logs could Anthony fix the real issue.

4-point no-errors policy

As a culture we as software engineers need to adopt a no errors/no warnings policy with strict adherence. More specifically:

1) Maintaining errors throughout a project is much simpler than just trying to fix it at the end.

2) How do you know you are not creating issues if you have constant errors and warnings? If the logs are 100% clear, you make a change, and then something occurs, you know you have caused a problem. If, however, the logs are full of problems, you are likely to be unaware of any issues you inject into a system with your work.

3) It might seem like more work but in the long run, but taking care of errors as you go creates a higher quality, more stable end product with less overall effort.

4) Lastly, nobody would expect a bridge builder to leave a site saying: “Well, the bridge works, but there are a few cracks here and there, but they shouldn’t matter.”

4 error-related answers you should know

If you understand the prior four points about errors and alarms, you will be able to answer the following questions quickly:

a) Where are system errors and warnings logged in my current application? [Database (DB) logs, state machine compiler (SMC), programmable logic controller (PLC) code compile warnings, etc.]

b) Do I have any errors?

c) If so, am I very clear on their meaning and have a clear plan to remove them? (Such as, I have three warnings because these duplicate coils exist, but I will remove them at the end of the day when I remove my simulation logic.)

d) If a peer were to sit down and review my work thoroughly for errors and warnings would I be proud of my work? (Here, think of the civil engineer building a bridge.)

The takeaway here is this: error messages matter. A lot. Consistently find them, track them, and remove them: NO errors, No Warnings!

- This blog aggregates expert advice from Callisto Integration, providing manufacturing consulting and systems integration. This blog provides integration advice in plant-floor controls, manufacturing execution systems (MES), and manufacturing consulting, from the factory floor through to the enterprise. Andrew Barker, P.Eng., Callisto Integration, compiled the advice. www.callistointegration.com 

Callisto Integration is a CSIA member as of 3/1/2015



No comments
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2015 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
Your leaks start here: Take a disciplined approach with your hydraulic system; U.S. presence at Hannover Messe a rousing success
Hannover Messe 2016: Taking hold of the future - Partner Country status spotlights U.S. manufacturing; Honoring manufacturing excellence: The 2015 Product of the Year Winners
Inside IIoT: How technology, strategy can improve your operation; Dry media or web scrubber?; Six steps to design a PM program
Getting to the bottom of subsea repairs: Older pipelines need more attention, and operators need a repair strategy; OTC preview; Offshore production difficult - and crucial
Digital oilfields: Integrated HMI/SCADA systems enable smarter data acquisition; Real-world impact of simulation; Electric actuator technology prospers in production fields
Special report: U.S. natural gas; LNG transport technologies evolve to meet market demand; Understanding new methane regulations; Predictive maintenance for gas pipeline compressors
Warehouse winter comfort: The HTHV solution; Cooling with natural gas; Plastics industry booming
Managing automation upgrades, retrofits; Making technical, business sense; Ensuring network cyber security
Designing generator systems; Using online commissioning tools; Selective coordination best practices

Annual Salary Survey

Before the calendar turned, 2016 already had the makings of a pivotal year for manufacturing, and for the world.

There were the big events for the year, including the United States as Partner Country at Hannover Messe in April and the 2016 International Manufacturing Technology Show in Chicago in September. There's also the matter of the U.S. presidential elections in November, which promise to shape policy in manufacturing for years to come.

But the year started with global economic turmoil, as a slowdown in Chinese manufacturing triggered a worldwide stock hiccup that sent values plummeting. The continued plunge in world oil prices has resulted in a slowdown in exploration and, by extension, the manufacture of exploration equipment.

Read more: 2015 Salary Survey

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
This article collection contains several articles on the vital role that compressed air plays in manufacturing plants.
This article collection contains several articles on the Industrial Internet of Things (IIoT) and how it is transforming manufacturing.
This article collection contains several articles on strategic maintenance and understanding all the parts of your plant.
click me