Two keys to measuring operational improvements

It’s not as easy as it seems, but getting good data can point to greater reliability.

By R. Keith Mobley, CMRP June 2, 2016

Believe it or not, tracking organizational performance is not as simple as it might sound. Many of us struggle with this dilemma, and few have determined an effective way to measure it. Our dilemma begins because the criteria that define performance vary depending on the level within the organization. For example, the organization level controls asset utilization—how the installed capacity will be used, and to a large degree, reliability. At the production-maintenance level, performance is measured as compliance to the business plan and its production schedule. Therefore, criteria used to measure performance must vary, but at the same time be consistent, so that measurements at the floor level can be rolled up to meet the criteria at the organizational level and conversely cascade down from organizational to floor-level measures. Once this is understood, there are two other issues to be resolved.

The first problem with measuring change is selecting good measurements. No one seems to be able to agree on the appropriate key performance indicators that define operations, maintenance, and reliability effectiveness. If you agree that a valid measurement is one that accurately defines controllable variables that are within the purview of a single function, then the options are limited. For example, what indicators could you use to measure maintenance effectiveness that are completely controlled by the maintenance functions? What about operations or reliability? There are few indicators that one can use to measure isolated functional performance. Because of the interdependence of these functions any attempt to measure them independently is quite difficult.

The best solution uses discrete performance indicators that measure critical performance variables at the lowest possible level, such as natural work teams on the factory floor, and progressively roles them up to parent functions, such as maintenance or operations, and ultimately to the complete organization.

For example, operational performance, at the work team level, should be measured on how well the team performs critical procedures, such as setup, startups, changeovers, etc. In this example, two considerations are essential—performance of a task versus a standard, and how consistently these tasks are performed. When these measurements are rolled up to the production and then to the plant level they provide an accurate measure of operational performance, as well as point out teams or groups of teams that are adversely affecting overall performance. In addition to adherence to operating standards, operational performance should also measure:

  • Cost per Unit Produced: the combination of labor and material costs consumed by operations, excluding direct materials, in the manufacturing or production process. It should include clear separation of the labor and material costs. Gaining maximum benefit from this measurement requires compiling data for each stock keeping unit (SKU), which is then rolled up to the composite value.
  • Labor cost per unit produced (by SKU).
  • Direct material cost per unit produced (by SKU).
  • Cost of Quality: a measure of the scrap, rework, and devalued losses created within the operations function. The total cost should include all incurred costs, such as lost revenue, additional materials handling, labor required to rework, incremental energy, and all other incurred cost resulting from off-spec or scrapped materials.
  • Production Output: a measure of the actual output of each production unit or system versus business plan output requirement. In recent years there has been much discussion surrounding how to measure production rate, but this measurement is restricted to the variables that are within the control of the operations function. In most cases, the business or production plan sets production rates that are well below design, and that issue must be addressed at a much higher level than operations.
  • Labor-hours per Unit Produced: a measure of the actual labor-hours per unit or units per labor-hour depending on industry. It measures the efficiency of the workforce in the conversion or manufacturing process. Adherence to standard hours required is a key measurement of manufacturing performance and is a prime source of losses in poor-performing organizations.
  • Energy Consumption per Unit Produced: Energy cost varies with the effectiveness and efficiency of the production process and is a significant variable cost that can be controlled through adherence to standards and best practices.

Maintenance effectiveness at the floor level can be measured in much the same manner. Compliance to standard time allocations for each maintenance tasks is a valid measure of the craftsperson’s and planner’s performance, but that means that all work must be well-planned, including accurate time standards. Acquiring performance data at the lowest level, e.g. individual task, provides the ability to roll up performance measurements to provide an accurate measure of overall maintenance performance and, subsequently, overall organizational performance. These measures should include:

  • Schedule Compliance: measures both mean-time-to-complete the tasks or activities and the interval or time when the activity was to take place. Because maintenance activities encompass a variety of types of work, such as preventive, proactive replacement, and corrective work, this criterion should be collected at the activity-type level and then rolled up to an overall compliance measure.
  • Resource Utilization: measures how effectively the craft labor pool is utilized and measures the actual productive time of the workforce. There are two best-practice standards for this measurement. The first, based on total craft workforce and used to measure utilization based on payroll hours or headcount, is 55%. The second benchmark, based on net available workforce, is 85%. Net available workforce is the labor hours that are actually available after deducting vacations, sick leave, light duty, etc.

As in production, there are an infinite number of parameters that can be measured, but only those that measure true performance are required:

  • Total Maintenance Cost per Unit Produced: This is the total labor and materials costs, including contractors, of the maintenance function. It is comprised of:
    • Direct Labor Costs: burdened labor cost of the crafts or technicians o Indirect Labor Costs: all other maintenance labor, such as planners, supervisors, clerical support, etc.
    • Contract Services: all contracted maintenance labor cost (contractor-supplied materials should be included in materials costs)
    • Materials Cost: the MRO materials that are used or consumed in the maintenance process and should not include inventory.

The business of reliability

Reliability, from a business perspective, is defined as processes, procedures, and practices that are consistent, waste-free, and cost-effective and is the simplest performance criterion to measure and track. The criteria used to measure operations and maintenance performance make up two-thirds of the total reliability performance evaluation.

When these values are rolled up to the organization level, they define the stability and consistency of execution performance. The missing third are those reliability drivers that are outside their combined span of control. These are performance drivers that are determined and controlled at the organizational level. These variables include:

  • Asset Utilization: measures how well the organization utilizes its installed capacity and is a critical performance measure for two reasons: 1) Arbitrary, organizational-level decisions to curtail planned production hours results in higher total cost of ownership; and 2) forcing production to meet delivery schedules and product mix with the reduced operating hours creates physical asset reliability issues, increases forced downtime, decreases useful life, and dramatically increases the total cost of ownership.
  • Cost of Goods Sold: measures the total cost, i.e. production and maintenance, required to manufacture or produce goods. It is compiled from the production and maintenance costs defined above, plus any incremental or additional costs created by other organizational functions.
  • Total Variable Costs: measures all costs that vary with performance or volume. It is compiled from values collected by operations and maintenance as defined above.
  • Safety: measures safety using the number of incidents and $/employee for Workers Compensation.

Without valid measurement parameters, improvement—even maintaining status quo—is difficult, but the problem does not stop there. Once meaningful key performance indicators are established, the next challenge is how to reliably acquire, interpret, and use this information. Too many performance measuring systems rely on arbitrary input from individuals throughout the organization without any assurance that the input is accurate. Short of full automation that removes humans from the process, the only way to improve accuracy is to clearly define the source, calculation, and meaning of each indicator as it applies to specific functions or areas of the organization.

These cannot be generic or open to interpretation. Perhaps the best approach is to train specific individuals within the organization to acquire raw data needed for the tracking program and enter it into an automated system that will then perform all calculations. At least this approach will remove some of the errors associate with manual generation of reports.

You cannot manage, and certainly cannot improve, what you cannot accurately measure. I am confident that today engineers are measuring tens if not hundreds of variables that are perceived to measure performance, but what are they really telling them about their performance? Reliability is the reciprocal of instability. It measures how stable an operation is.

Organizational performance is how well engineers are using their installed capacity and human resources to meet market demand. And maintenance measures show how well resources are used to provide value-added activities that will sustain asset reliability into the future. How are you doing? Do your existing measurements address these definitions? If not, why not?

—Keith Mobley has more than 50 years of direct experience in corporate management, process optimization, and reliability engineering. For the past 25 years, he has helped hundreds of clients worldwide achieve and sustain world-class performance. He can be reached at