Trending and AI

What is AI Missing?

I was reading a claim that AI could scan a database and accurately trend root cause analysis data. That statement made me think of a comment that I think was made by Dr. Donald J. Wheeler (a statistical process control expert and trending guru). He said something to the effect of:

You need specific process knowledge if you want to learn from trends.

His idea made me question AI and trending because AI might not have specific process knowledge.

To explain this, let me use two examples…

Example 1: Change in Statistics

We were hired to look at seven years of root cause analysis data and see if we could find any trends that the company might have missed. We ran several XmR graphs of the data and found a surprising statistic. At about the end of the third year of data, the normal number of root causes per incident changed from about 4.5 to 1. The single root cause per incident continued for about two years. Then, just as suddenly, the statistic changed again and went back up to about 4 root causes per incident.

What happened? I asked our contact for some specific knowledge of the process. He told me that a new manager took over, and while he was in charge, he said there could only be one root cause per incident. Thus, the investigators were told to pick the “rootiest” of the root causes and discard the others when recording the results of their investigation.

We concluded that we could not trust the data from those two years because of the arbitrary nature of the discarding of root cause data.

Would AI have detected this anomaly, or would it look for trends in the root causes without understanding how the system had changed?

Example 2: From Human Error to Equipment Failure

At a company, they conducted an annual review of their incident trends. For one year, they drew a Pareto Chart of their causes, and “Human Error” was the preeminent cause by a large margin. The next year, the data had changed. That second year, the year’s incident investigations data showed that “Equipment Failure” was by far the biggest cause of incidents.

It was obvious to management that they needed a new effort to improve equipment reliability.

But my first thought was:

What had changed?

Did they implement an amazing human performance improvement program, and only equipment failures were left to correct? Had something changed in their maintenance program? How did the number of incidents change?

Here was the interesting change…

In the first year, investigators had coded root causes as human errors. They found that people were being punished for making mistakes. These were their friends and co-workers. So, after the first year review, they started looking for and coding root causes as equipment failure. The equipment didn’t get blamed, and their friends weren’t punished.

I’m sure that AI would have detected the change from human error to equipment failure. But would AI uncover the reason for the change? Would AI be able to collect honest answers from the investigators about why the change occurred?

Even more important, could AI tell the company that human error is not a root cause? Could they advise the company that their investigators need a better root cause analysis system? Could AI explain to management that blame can cause unexpected outcomes in incident investigations and root cause analysis, and that they ought to adopt an “Opportunity to Improve Vision“?

Conclusion

AI can detect trends in data, but it may not have the specific process knowledge needed to understand trends or recommend effective corrective action.

What do you think? Can AI overcome these limitations? Is AI trending an effective management tool? Or is the human touch still needed?