Do not ignore those recurring errors in manufacturing or processing

Error messages matter, a lot. Find them, track them, and remove them. Adopt a 4-point no-errors policy. Can you answer 4 questions about errors, warnings, and alarms in manufacturing and processing? Learn from Anthony Baker.

07/26/2013


You know the manufacturing automation or process control errors we are talking about: The nagging errors in the log that don’t seem to go away. The system is “running fine” but there’s a tag in the PLC or other controller that doesn’t exist, or a link in the application that doesn’t compile completely. The lesson here is simple, but one that lots of engineers ignore:

NO ERRORS NO WARNINGS.

No errors, no warnings, advises Callisto IntegrationAnthony Baker received a recent call from a very stressed out, very important client. The client had an human-machine interface (HMI) software system that would become unresponsive after 12-15 hours of uptime. The onsite engineers were unsure of the cause and had called Anthony as a last resort. The system was in a fragile state, and production was at risk.

This was a new site for Anthony and, despite having a deep understanding of this particular application, Anthony did what lots of engineers do and went into the application error logs, created a list of errors and one-by-one hunted down the root cause and eliminated the message. In many cases the error was a warning such as “Tagname ZZ does not exist in the PLC” or “Database connection Y has timed out.” All the errors could have been prevented with clean implementation and more thorough test plans that include a “NO ERRORS NO WARNINGS” policy.

Eliminating the errors in the logs one by one, the issue was eventually solved. As a note, the most frequent and obvious error was actually not the root cause of the outage — it was a subtle, one-time issue that cropped up in the logs and was dutifully logged by the HMI application. Unfortunately, the root-cause error was drowned out by a recurring issue that had been present in the system for years. Only by fixing this error flooding the logs could Anthony fix the real issue.

4-point no-errors policy

As a culture we as software engineers need to adopt a no errors/no warnings policy with strict adherence. More specifically:

1) Maintaining errors throughout a project is much simpler than just trying to fix it at the end.

2) How do you know you are not creating issues if you have constant errors and warnings? If the logs are 100% clear, you make a change, and then something occurs, you know you have caused a problem. If, however, the logs are full of problems, you are likely to be unaware of any issues you inject into a system with your work.

3) It might seem like more work but in the long run, but taking care of errors as you go creates a higher quality, more stable end product with less overall effort.

4) Lastly, nobody would expect a bridge builder to leave a site saying: “Well, the bridge works, but there are a few cracks here and there, but they shouldn’t matter.”

4 error-related answers you should know

If you understand the prior four points about errors and alarms, you will be able to answer the following questions quickly:

a) Where are system errors and warnings logged in my current application? [Database (DB) logs, state machine compiler (SMC), programmable logic controller (PLC) code compile warnings, etc.]

b) Do I have any errors?

c) If so, am I very clear on their meaning and have a clear plan to remove them? (Such as, I have three warnings because these duplicate coils exist, but I will remove them at the end of the day when I remove my simulation logic.)

d) If a peer were to sit down and review my work thoroughly for errors and warnings would I be proud of my work? (Here, think of the civil engineer building a bridge.)

The takeaway here is this: error messages matter. A lot. Consistently find them, track them, and remove them: NO errors, No Warnings!

- This blog aggregates expert advice from Callisto Integration, providing manufacturing consulting and systems integration. This blog provides integration advice in plant-floor controls, manufacturing execution systems (MES), and manufacturing consulting, from the factory floor through to the enterprise. Andrew Barker, P.Eng., Callisto Integration, compiled the advice. www.callistointegration.com 



No comments
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2013 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
The true cost of lubrication: Three keys to consider when evaluating oils; Plant Engineering Lubrication Guide; 11 ways to protect bearing assets; Is lubrication part of your KPIs?
Contract maintenance: 5 ways to keep things humming while keeping an eye on costs; Pneumatic systems; Energy monitoring; The sixth 'S' is safety
Transport your data: Supply chain information critical to operational excellence; High-voltage faults; Portable cooling; Safety automation isn't automatic
Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Plant Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.

Maintaining low data center PUE; Using eco mode in UPS systems; Commissioning electrical and power systems; Exploring dc power distribution alternatives
Synchronizing industrial Ethernet networks; Selecting protocol conversion gateways; Integrating HMIs with PLCs and PACs
Why manufacturers need to see energy in a different light: Current approaches to energy management yield quick savings, but leave plant managers searching for ways of improving on those early gains.

Annual Salary Survey

Participate in the 2013 Salary Survey

In a year when manufacturing continued to lead the economic rebound, it makes sense that plant manager bonuses rebounded. Plant Engineering’s annual Salary Survey shows both wages and bonuses rose in 2012 after a retreat the year before.

Average salary across all job titles for plant floor management rose 3.5% to $95,446, and bonus compensation jumped to $15,162, a 4.2% increase from the 2010 level and double the 2011 total, which showed a sharp drop in bonus.

2012 Salary Survey Analysis

2012 Salary Survey Results

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.