Not statistically significant does not mean insignificant

Do you have customer data that you're not actively using to improve your product?

By Rebecca A. Morgan, Fulcrum ConsultingWorks Inc. August 26, 2016

My fully loaded 2012 Audi A6 had an intermittent frustrating problem since the day I bought it.

No diagnostic codes indicated a problem. Escalation to German engineering had me ready to move back to Lexus. Their response was, "It must not really be happening. Our codes would indicate if it were."

That obnoxious response was based on the assumption they had thought of every cause of failure in developing the diagnostic codes. Failure mode effects analysis (FMEA) is not 100% and never will be.

Do you have customer data that you’re not actively using to improve your product?

Four years after I first reported the issue, Audi issued an urgent safety recall for the problem that I had been experiencing.

Why the delay? It only happens on fully loaded vehicles with air conditioned seats. They say it took them four years because of the limited number of complaints. But you and I both know that even low-level interest and basic statistical analysis could have identified correlation and subsequently led to root-cause analysis.

Is your company only examining customer complaints when they reach a certain established quantity?

Ten may not be a statistically valid sample size, but it may well be enough to identify potential correlations. Twenty may be enough to narrow in. Don’t let the statisticians (I have a PhD-level econometrics background myself) talk you out of recognizing and dealing with problems.

Excellent companies look at problems from the standpoint of their customer and with the assumption that their buyers are not crazy. While a reported problem may affect less than .01% of your sales, it’s 100% to the customer who has it.

Saying you care about quality is only believable if you demonstrate that to each customer. Data collection only has value if you use it. Letting data age until a certain "critical mass" is reached is not the mark of a company committed to operational excellence. Neither is technical resources implementing "emissions-cheating software" on a very statistically significant number of vehicles. Oh! Unethical behavior also precludes a commitment to excellence. Mistakes happen. Fraud is not a mistake.

Data can tell a fascinating story, if you look at it.

-Rebecca A. Morgan is an AME author and president at Fulcrum ConsultingWorks Inc. This article originally appeared on Association for Manufacturing Excellence is a CFE Media content partner.

Original content can be found at