Support Vector Machine (SVM)

What is Support Vector Machine?

Support Vector Machine are supervised machine learning algorithms used mainly for classification and regression tasks. If a SVM is used for classification, it’s called Support Vector Classifier (SVC). Similarly, for regression it’s called Support Vector Regressor (SVR)

Where is SVM used?

SVM can be used wherever we use other machine learning techniques such as Logistic Regression, Decision Trees, Linear Regression, Naive Bayes Classifier etc. However, SVM may be particularly more suitable for following cases-

Sparse data
High Dimensional data
Text Classification
Data is nonlinear
Image classification
Data has complex patterns
Etc.

How does an SVM work?

A support vector machine, works to separate the pattern in the data by drawing a linear separable hyperplane in high dimensional space. For example in the 2D image below, we need to separate the green points from the red points. We can draw many hyperplanes such as H1, H2, H3 and H4. They all help in separating the points.

However, since there are many possible hyperplanes as denoted in the image below, which hyperplane should be chosen? The answer is- the plane which maximizes the separation between the green and the red points. In this case, it happens to be H3.

What happens if the data is not linearly separable?

Kernel trick or Kernel function helps transform the original non-linearly separable data into a higher dimension space where it can be linearly transformed. See image below-

What is the best hyperplane?

As we discussed earlier, the best hyperplane is the one that maximizes the distance (you can think about the width of the road) between the classes as shown below