Congruence Bias, Systems Administrators, and Troubleshooting

90% of Systems Administration is the art of troubleshooting. “Art” because most systems administrators I know practice troubleshooting through inspiration rather than deductive insight. It often isn’t a matter of assessing all possible evidence about a problem and then deducing the answer. Generally most systems administrators (and I’m guilty of this myself) jump to a conclusion about the cause of a problem based on some of the clues we see. We don’t get time to see all the clues because we’ve taken the first few and used them to rapidly come to a conclusion. Sometimes through dumb luck that conclusion is correct. Most often it’s wildly off base and it takes some time before we hit upon the real cause. The chances are that the first clues you see aren’t going to be all relevant to the problem you are trying to diagnose.

Congruence Bias is a problem in diagnostic reasoning where people who believe they have discovered a solution to a problem tend to ask questions that reinforce the initial hypothesis rather than asking questions that might lead to the hypothesis being rejected. Put another way, when we troubleshoot, we look for evidence that our hypothesis is correct rather than looking for evidence that our hypothesis is incorrect. This is a problem because of another type of bias, known as confirmation bias. As we attempt to confirm our supposition, we tend to ignore anything that might conflict with that idea. We miss the clues that contradict the conclusion that we jumped to.

Consider the following. You have been banging your head against a problem for some time. You thought you had the answer, but when you try to solve the problem using that answer, the problem remains. Only after you’ve given up do you notice a small but significant detail that leads you to completely reassess your solution. You might even say to yourself “well if I’d seen that a couple of hours ago, I wouldn’t have ended up this blind alley”. You would have seen those clues if you hadn’t already jumped to an incorrect conclusion. It’s only when you give up that conclusion to you see the evidence with fresh eyes.

The reality is that because you were attempting to directly test your hypothesis, you didn’t attempt an indirect test. Because you attempted a test to confirm the answer you arrived at through inspiration, you missed performing the test that would have discounted that answer.

In terms of troubleshooting, a good place to start is to assume that the first few things that come into your head are probably random and that while you are seeing clues, the first ones you see are probably irrelevant. While sometimes you will be right when you jump to a conclusion, you are more likely to solve a problem through methodical deduction and evidence gathering than you are by relying upon inspiration to deposit the answer directly into your head.

You can find out more about congruence bias by reading the following Wikipedia article:

http://en.wikipedia.org/wiki/Congruence_bias