The Two Variations of Confusion Matrix
Memory hacks to remember its two variations and associated formulae
You may already know that there are two schools of thought (wiki vs python) on how to display a confusion matrix.
For every beginner, there comes a time when one sees the ‘other’ variation and tilts one’s head to make sense of it. The literature supports one layout while the code seems to spew out the other.
To understand the basics of Confusion Matrix, you can have a look here.
Here are the variations:
as compared to:
It’s not a case of simply interchanging the rows and columns, but also the fact that True Positives which were at top-left corner are now replaced by True Negatives.
If you are using built-in scikit-learn functions like confusion_matrix, you need to be aware of the input and output layout, else all your calculations of Sensitivity (Recall), Specificity, Precision, F1 score are going to be wrong.
Show me some proof!
Okay, let’s have an actual example to figure this out, through code.
This is our dataset of medical results of 17 people. They are either unwell (Positive test result) or healthy (Negative test result). The first column shows their actual state while the second column is the one predicted by our algorithm. The code used to generate this:
Perspective 1:
If like me, you have studied most of statistics via books and publications rich in examples from the medical field, the Actuals (or the Gold Standard) will be on the top as columns and True Positive in the top-left cell. Our dataset will look like this:
Here’s the python code to generate this:
cm = confusion_matrix(data.Predicted, data.Actual, labels=[‘Positive’, ‘Negative’])print(cm)
Note how we pass data.Predicted first and data.Actual later. Also the order of labels is Positive first.
Output:
My memory hack to remember the order of arguments:
I just read it the way one reads English — left to right. And I’m positive this is the layout one ought to use.
To calculate all the metrics, one can use sklearn’s classification_report function.
I love this function because the order of Actual and Predicted arguments remains the same, no matter what your perspective is. All you need to specify differently is the order of labels.
report = classification_report(data.Actual, data.Predicted, labels=[‘Positive’, ‘Negative’])print(report)
It’s always Actual first, Predicted later — alphabetically arranged, elegantly. The output it generates is:
Perspective 2:
Those who have learnt it through programming documentation and tutorials hate the above representation. It’s always the other way round!
The position of False Positives and False Negatives remain the same in a 2x2 confusion matrix, whichever perspective you choose.
The memory hack remains the same, read from left to right but this time give more importance to Negatives by mentioning them first in the labels argument.
cm2 = confusion_matrix(data.Actual, data.Predicted, labels=[‘Negative’, ‘Positive’])print(cm2)
And the good old classification_report remains the same, the only change being in the order of labels:
report2 = classification_report(data.Actual, data.Predicted, labels=[‘Negative’, ‘Positive’])print(report2)
There you are, both the results perfectly in sync now!
Additional Tip — Never Forget the Formulae:
I often had to look up my notes to remember what Precision meant, or what was the formula for Sensitivity (also known as Recall). So, here’s a memory hack I use to remember them.
First, the actual formulae for them, so that you can relate easily. I explain here on the basis of Perspective 1, but you can change it very easily for perspective 2 if you wish to.
I like to arrange them in an alphabetical order first, so that I don’t mix them up — 1. Precision 2. Recall 3. Specificity
Precision = TP/(TP+FP)
Recall = TP/(TP+FN)
Specificity = TN/(TN+FP)
My memory hack to remember them:
Precision is the first cell divided by the row sum: 2/6 = 0.33
Recall is the first cell divided by the column sum: 2/7 = 0.29
Specificity is the last cell divided by the column sum: 6/10 = 0.60
That’s it! Made my life a little bit easier, hope it helps you too. Have you got any tips and tricks which work for you? Do let me know in the comments section or drop me a line separately.
Interested in sharing ideas, asking questions or simply discussing thoughts? Connect with me on LinkedIn, YouTube, GitHub or through my website: I am Just a Student.
See you around & happy learning!
