Two keys to measuring operational improvements

It’s not as easy as it seems, but getting good data can point to greater reliability.


Among the keys to tracking operational excellence are selecting good measurements then inforcing standards. Images courtesy: CFE Media.Believe it or not, tracking organizational performance is not as simple as it might sound. Many of us struggle with this dilemma, and few have determined an effective way to measure it. Our dilemma begins because the criteria that define performance vary depending on the level within the organization. For example, the organization level controls asset utilization—how the installed capacity will be used, and to a large degree, reliability. At the production-maintenance level, performance is measured as compliance to the business plan and its production schedule. Therefore, criteria used to measure performance must vary, but at the same time be consistent, so that measurements at the floor level can be rolled up to meet the criteria at the organizational level and conversely cascade down from organizational to floor-level measures. Once this is understood, there are two other issues to be resolved.

The first problem with measuring change is selecting good measurements. No one seems to be able to agree on the appropriate key performance indicators that define operations, maintenance, and reliability effectiveness. If you agree that a valid measurement is one that accurately defines controllable variables that are within the purview of a single function, then the options are limited. For example, what indicators could you use to measure maintenance effectiveness that are completely controlled by the maintenance functions? What about operations or reliability? There are few indicators that one can use to measure isolated functional performance. Because of the interdependence of these functions any attempt to measure them independently is quite difficult.

The best solution uses discrete performance indicators that measure critical performance variables at the lowest possible level, such as natural work teams on the factory floor, and progressively roles them up to parent functions, such as maintenance or operations, and ultimately to the complete organization.

For example, operational performance, at the work team level, should be measured on how well the team performs critical procedures, such as setup, startups, changeovers, etc. In this example, two considerations are essential—performance of a task versus a standard, and how consistently these tasks are performed. When these measurements are rolled up to the production and then to the plant level they provide an accurate measure of operational performance, as well as point out teams or groups of teams that are adversely affecting overall performance. In addition to adherence to operating standards, operational performance should also measure:

  • Cost per Unit Produced: the combination of labor and material costs consumed by operations, excluding direct materials, in the manufacturing or production process. It should include clear separation of the labor and material costs. Gaining maximum benefit from this measurement requires compiling data for each stock keeping unit (SKU), which is then rolled up to the composite value.
  • Labor cost per unit produced (by SKU).
  • Direct material cost per unit produced (by SKU).
  • Cost of Quality: a measure of the scrap, rework, and devalued losses created within the operations function. The total cost should include all incurred costs, such as lost revenue, additional materials handling, labor required to rework, incremental energy, and all other incurred cost resulting from off-spec or scrapped materials.
  • Production Output: a measure of the actual output of each production unit or system versus business plan output requirement. In recent years there has been much discussion surrounding how to measure production rate, but this measurement is restricted to the variables that are within the control of the operations function. In most cases, the business or production plan sets production rates that are well below design, and that issue must be addressed at a much higher level than operations.
  • Labor-hours per Unit Produced: a measure of the actual labor-hours per unit or units per labor-hour depending on industry. It measures the efficiency of the workforce in the conversion or manufacturing process. Adherence to standard hours required is a key measurement of manufacturing performance and is a prime source of losses in poor-performing organizations.
  • Energy Consumption per Unit Produced: Energy cost varies with the effectiveness and efficiency of the production process and is a significant variable cost that can be controlled through adherence to standards and best practices.

Maintenance effectiveness at the floor level can be measured in much the same manner. Compliance to standard time allocations for each maintenance tasks is a valid measure of the craftsperson's and planner's performance, but that means that all work must be well-planned, including accurate time standards. Acquiring performance data at the lowest level, e.g. individual task, provides the ability to roll up performance measurements to provide an accurate measure of overall maintenance performance and, subsequently, overall organizational performance. These measures should include:

  • Schedule Compliance: measures both mean-time-to-complete the tasks or activities and the interval or time when the activity was to take place. Because maintenance activities encompass a variety of types of work, such as preventive, proactive replacement, and corrective work, this criterion should be collected at the activity-type level and then rolled up to an overall compliance measure.
  • Resource Utilization: measures how effectively the craft labor pool is utilized and measures the actual productive time of the workforce. There are two best-practice standards for this measurement. The first, based on total craft workforce and used to measure utilization based on payroll hours or headcount, is 55%. The second benchmark, based on net available workforce, is 85%. Net available workforce is the labor hours that are actually available after deducting vacations, sick leave, light duty, etc.

As in production, there are an infinite number of parameters that can be measured, but only those that measure true performance are required:

  • Total Maintenance Cost per Unit Produced: This is the total labor and materials costs, including contractors, of the maintenance function. It is comprised of:

    • Direct Labor Costs: burdened labor cost of the crafts or technicians o Indirect Labor Costs: all other maintenance labor, such as planners, supervisors, clerical support, etc.
    • Contract Services: all contracted maintenance labor cost (contractor-supplied materials should be included in materials costs)
    • Materials Cost: the MRO materials that are used or consumed in the maintenance process and should not include inventory.

The business of reliability

Reliability, from a business perspective, is defined as processes, procedures, and practices that are consistent, waste-free, and cost-effective and is the simplest performance criterion to measure and track. The criteria used to measure operations and maintenance performance make up two-thirds of the total reliability performance evaluation.

When these values are rolled up to the organization level, they define the stability and consistency of execution performance. The missing third are those reliability drivers that are outside their combined span of control. These are performance drivers that are determined and controlled at the organizational level. These variables include:

  • Asset Utilization: measures how well the organization utilizes its installed capacity and is a critical performance measure for two reasons: 1) Arbitrary, organizational-level decisions to curtail planned production hours results in higher total cost of ownership; and 2) forcing production to meet delivery schedules and product mix with the reduced operating hours creates physical asset reliability issues, increases forced downtime, decreases useful life, and dramatically increases the total cost of ownership.
  • Cost of Goods Sold: measures the total cost, i.e. production and maintenance, required to manufacture or produce goods. It is compiled from the production and maintenance costs defined above, plus any incremental or additional costs created by other organizational functions.
  • Total Variable Costs: measures all costs that vary with performance or volume. It is compiled from values collected by operations and maintenance as defined above.
  • Safety: measures safety using the number of incidents and $/employee for Workers Compensation.

Without valid measurement parameters, improvement—even maintaining status quo—is difficult, but the problem does not stop there. Once meaningful key performance indicators are established, the next challenge is how to reliably acquire, interpret, and use this information. Too many performance measuring systems rely on arbitrary input from individuals throughout the organization without any assurance that the input is accurate. Short of full automation that removes humans from the process, the only way to improve accuracy is to clearly define the source, calculation, and meaning of each indicator as it applies to specific functions or areas of the organization.

These cannot be generic or open to interpretation. Perhaps the best approach is to train specific individuals within the organization to acquire raw data needed for the tracking program and enter it into an automated system that will then perform all calculations. At least this approach will remove some of the errors associate with manual generation of reports.

You cannot manage, and certainly cannot improve, what you cannot accurately measure. I am confident that today engineers are measuring tens if not hundreds of variables that are perceived to measure performance, but what are they really telling them about their performance? Reliability is the reciprocal of instability. It measures how stable an operation is.

Organizational performance is how well engineers are using their installed capacity and human resources to meet market demand. And maintenance measures show how well resources are used to provide value-added activities that will sustain asset reliability into the future. How are you doing? Do your existing measurements address these definitions? If not, why not?

—Keith Mobley has more than 50 years of direct experience in corporate management, process optimization, and reliability engineering. For the past 25 years, he has helped hundreds of clients worldwide achieve and sustain world-class performance. He can be reached at

Top Plant
The Top Plant program honors outstanding manufacturing facilities in North America.
Product of the Year
The Product of the Year program recognizes products newly released in the manufacturing industries.
System Integrator of the Year
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
September 2018
2018 Engineering Leaders under 40, Women in Engineering, Six ways to reduce waste in manufacturing, and Four robot implementation challenges.
GAMS preview, 2018 Mid-Year Report, EAM and Safety
June 2018
2018 Lubrication Guide, Motor and maintenance management, Control system migration
August 2018
SCADA standardization, capital expenditures, data-driven drilling and execution
June 2018
Machine learning, produced water benefits, programming cavity pumps
April 2018
ROVs, rigs, and the real time; wellsite valve manifolds; AI on a chip; analytics use for pipelines
Spring 2018
Burners for heat-treating furnaces, CHP, dryers, gas humidification, and more
August 2018
Choosing an automation controller, Lean manufacturing
September 2018
Effective process analytics; Four reasons why LTE networks are not IIoT ready

Annual Salary Survey

After two years of economic concerns, manufacturing leaders once again have homed in on the single biggest issue facing their operations:

It's the workers—or more specifically, the lack of workers.

The 2017 Plant Engineering Salary Survey looks at not just what plant managers make, but what they think. As they look across their plants today, plant managers say they don’t have the operational depth to take on the new technologies and new challenges of global manufacturing.

Read more: 2017 Salary Survey

The Maintenance and Reliability Coach's blog
Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
One Voice for Manufacturing
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Maintenance and Reliability Professionals Blog
The Society for Maintenance and Reliability Professionals an organization devoted...
Machine Safety
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
Research Analyst Blog
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Marshall on Maintenance
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
Lachance on CMMS
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
Material Handling
This digital report explains how everything from conveyors and robots to automatic picking systems and digital orders have evolved to keep pace with the speed of change in the supply chain.
Electrical Safety Update
This digital report explains how plant engineers need to take greater care when it comes to electrical safety incidents on the plant floor.
IIoT: Machines, Equipment, & Asset Management
Articles in this digital report highlight technologies that enable Industrial Internet of Things, IIoT-related products and strategies.
Randy Steele
Maintenance Manager; California Oils Corp.
Matthew J. Woo, PE, RCDD, LEED AP BD+C
Associate, Electrical Engineering; Wood Harbinger
Randy Oliver
Control Systems Engineer; Robert Bosch Corp.
Data Centers: Impacts of Climate and Cooling Technology
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
Safety First: Arc Flash 101
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
Critical Power: Hospital Electrical Systems
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
Design of Safe and Reliable Hydraulic Systems for Subsea Applications
This eGuide explains how the operation of hydraulic systems for subsea applications requires the user to consider additional aspects because of the unique conditions that apply to the setting
click me