Improve cost estimating accuracy

How many times have you used the term "plus or minus 10%" in reference to a cost estimate? Most of us who are in the project world have. Have you ever stopped to think what it really means? In its common engineering usage, "plus or minus X%" is a nearly meaningless phrase inserted after a dollar amount to meet some company criteria. Understanding the true meaning of estimating accuracy, and having a method to make more useful estimates, is valuable to bring meaning to real costs.

If you state that an estimate is accurate to any particular level without a corresponding confidence level, either written with the accuracy statement or part of policy, renders the term truly meaningless. A $1 million estimate accurate to ±10% and at a 90% confidence level is much different than one accurate to ±10% at a 10% confidence level. In the first case, the expectation is that is will be right 9 times out of 10; in the latter, it will be wrong 9 times out of 10.

In manufacturing, each project is unique, and we have relatively few projects. This includes both the smaller projects typically handled by plant engineering organizations as well as the larger projects (in contrast, for example, to a fast food chain that churns out many copies of the same thing every year). We just don't have a sufficient amount of reliable data to allow a statistically valid analysis to be done.

This is a "Catch 22" if there ever was one. On one hand, our estimate accuracies are invalid without stating confidence, and on the other, we cannot determine confidence. This situation tends to confirm the point made earlier that "in its common engineering usage, 'plus or minus X%' is a nearly meaningless phrase inserted after a dollar amount." The question then becomes how to deal with this conundrum. Here are four suggestions:

Let's face it; we are not going to get rid of the terms. Accountants and managers are often not statisticians, and a term like "plus or minus 10%" is just verbal shorthand, and needs to be viewed in that way.

By establishing two or three standard, stated accuracies, we can assign expected deliverables and degrees of due diligence that will mark acceptance of the stated accuracy. A typical spread would look something like this, but your company can establish any values it wishes and is willing to accept the risks for it.

• Rough Order of Magnitude: ±50%
• Conceptual: ±25%
• Capital Grade: ±10%

While we can't establish a meaningful confidence level, we can establish a list of required deliverables and an expected level of due diligence done on each of those deliverables for each level of estimating accuracy. Various estimating books have suggestions for what deliverables and to what level they are required, but that should be merely a starting point.

In your company, engineering and management must work together to define both the list and the level of due diligence. The basic statement is: "Do these things, do them well, and the project manager will be confident in the estimate, and management will be willing to accept the attendant risk.

There are some tradeoffs that are inescapable. For example, more deliverables and more due diligence mean more time and more upfront cost. Fewer deliverables and less due diligence translate directly to higher risk that the financial goals will not be met, but have a lower expense to estimate.

Done properly, there is an alignment between the requirements set forth and the accuracy claimed. It is also possible (and all to frequent) to have a misalignment. Typically, it is in the form of too little work for too much stated accuracy in the cause of cost reduction (i.e., keeping estimating expenditures to a minimum).

We have all used this verbal shorthand so long and so loosely that true understanding of what we mean has become lost. We need to clearly communicate to everyone who sees and uses one of our estimates that 50% of our projects will overrun (the plus side) and 50% underrun (the minus side). Also some percentage will exceed the limits.

In the "confidence" discussion earlier, we talked about a ±10% estimate to a 90% confidence level. Using this example, 5% of our projects will be under the -10% limit, 45% under budget but by less than 10%, 45% over budget by less than 10% and 5% over budget by more than 10%. (See Fig. 1.)

We should do "lessons learned," especially on projects outside the limits, but barring excessive percentages or demonstrable poor performance, we can accept it as a natural part of the cycle.

Let's look at what happens using these basic assumptions for the same project being run in two different environments. Alignments between funding guidelines and estimating for the two cases are shown in Fig. 2.

The project's base cost estimate is $1,000,000 in both cases.

Also in both cases there is a personal performance issue for the project manager if there is an overage of more than 10%.

In Company A, the due diligence and deliverables align with the stated ±10%.

In Company B, the due diligence aligns with ±25% for a stated 10% overage.

Let's dissect what happened. We started at the same point — a million dollar job. Project manager A spent 5% of the "real cost" for a "true" ±10% estimate while project manager B saved $35,000 by spending only 1.5% for a "true" ±25%. Both will then add contingency, but B will add more. B has more known and unknown holes. Then B will make one more correction A doesn't have to. He will make a correction based on his accuracy. Chances are project manager B will be able to spend his way into compliance with company financial guidelines and a good performance review.

Contingency and estimate accuracy are often confused. In our example, it looks like project manager B is "double dipping." In fact, he is not. Contingency is added to an estimate to account for unknowns. The more you know, the more confident you are you have your bases covered, and the lower the contingency. In these cases, both project managers honestly evaluated the risks, and B assigned a contingency accordingly larger. Contingency is a risk modifier used to bring the cost estimate to the range midpoint.

Estimate accuracy reflects the statistical probability that the estimate will come within stated parameters. It reflects the width of the bell curve around the midpoint. In PM A's case A need not add anything; required accuracy and confidence in the estimate are aligned. If B's requirement were +/-25% nothing would need to be added there.

However there is a misalignment of accuracy as supported by the deliverables and the requirements of the company. So money is added to make the estimate a +10% (in guideline) -40% (but we won't talk about that). In fact, we move the estimate off from the mid point to secure the high end.

Alignment between deliverables and diligence with stated accuracy must be maintained for optimum cost performance. Since we don't have statistical proof to back this up, project managers must believe it to be true (project manager B was really acting on perception of reality in adding the extra 15%), and management must be willing to either support the effort or accept the risk.

Following these types of steps should improve estimating and cost performance. But after implementation, you will need to assess both the required deliverables and the required level of due diligence. Some deliverables may need to be added. Others will prove of little or no value and should be dropped. The degree of due diligence in creating the deliverables should be adjusted based on experience. In all of this, we are dealing with estimates which by their nature are inaccurate.

We cannot eliminate the uncertainties, but we can manage them. And by managing them, we assure successful projects and, by extension, successful project managers.

Randall Lund can be reached for more information on capital project management at 269-372-9240 or rdlund@ix.netcom.com. Article edited by Executive Editor Richard L. Dunn, 815-236-2196, or dunncomm@comcast.net.