In this post, we are going to learn about multi-class classification.
We have learned about linear and logistic regression in our earlier posts.
If you are not familiar with the algorithms check out the links below.
Full Guide on Machine Learning
Before starting the article go through the overview given below
Overview
- What is the classification?
- Formula for classification
- Implementation in python.
So let’s start with the definition of classification.
What is the classification?
Classification is a part of a supervised machine learning algorithm
And it is used to output discrete values.
The real-world example of classification are:
spam-detection or,
detecting a breed of a dog.
In simple classification, we train our model to output binary values such as 0 or 1, true or false, yes or no.
In this approach, we have more than two categories.
y = {0,1,2,….,n}
We divide the problem in n+1 binary classification problem and then apply binary logistic regression
So now let’s derive the formula for the multi-class classification with the use of binary logistic regression.
For every value of y, we will it into one class and put all the other values in the second class.
Here is the formula of hypothesis function for n values of y.
We take n hypothesis function for n values of y and perform binary logistic regression.
To learn about binary Logistic Regression click the link.
Now let’s implement the algorithm using Python.
Here we will use the class problem of detecting the handwritten number.
I hope that you have the basic knowledge of python and it’s library such as sci-kit learn, numpy, etc.
For reference use the links below:
Seems like you are ready to implement it. So let’s dive into it.
Implementation
First of all, we will import our libraries.
Then we will import the Logistic Regression Class from sklearn.
%matplotlib inline
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
digits = load_digits()
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(digits.data,digits.target, test_size=0.2)
Now we will predict our values.
model.predict(digits.data[0:5])
After predicting the values we will create a confusion matrix.
So you are thinking about what is confusion matrix.
It is used to describe the performance of the classifier.
That means it shows how well our classifier has predicted the values.
y_predicted = model.predict(X_test)
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_predicted)
cm
Now we will plot our confusion matrix using the seaborn library.
So we can easily see how well our model has predicted the values.
import seaborn as sn
plt.figure(figsize = (10,7))
sn.heatmap(cm, annot=True)
plt.xlabel('Predicted')
plt.ylabel('Truth')
So from this figure, we can see that our model had predicted some values that we feed.
The values we feed our model is on the x-axis.
And the actual values are on the y-axis.
Here we can see that I have circle two different sets of values with two different colors.
The white color show the wrong prediction.
In the white circle, we feed our model with the image of 1 but it predicted that it’s 8.
The number 2 in the white circle shows how many times our model predicted that value.
Similarly, the yellow circle shows the correct prediction of the digits.
The numbers inside the yellow circle show that how many times our model predicted the right value.
Conclusion
In this post, we discussed what is classification.
We looked into what is multi-class classification.
After that, we derived the formula for our algorithm.
Finally, we implemented the algorithm using python and it’s a library.
Check out my post on Logistic regression
https://learn-ml.com/index.php/2019/05/29/logistic-regression-step-by-step-guide-in-python/
