Vapnik–Chervonenkis(VC) dimensions

A theoretical way to understand which model is powerful.

Photo by Claire Zhu on Unsplash

VC dimensions are used to quantify how powerful is the model

For example, if we have a pre-processed data set and we have Linear regression, SVM, Ensemble models, Neural Network models.

In a real-world(Practical) what we do is apply one by one model on given data set and find the accuracy of each model. The model gives the highest accuracy that is the powerful model.

But when comes to statistical Machine learning VC dimensions used to find which model is powerful

VC dimension of a model = Maximum no of points that can be separated by a model for all possible configuration

Ex: Linear models( Linear regression, linear SVM )

3 points:

Image source: http://me-ramesh.blogspot.com/p/machine-learning.html

Three points are like any way we always have a possible way to separate them (Classify them)

4 points:

Image Source: http://me-ramesh.blogspot.com/p/machine-learning.html

When coming to four points we are not able to separate all possible configurations.

According to VC dimension definition

VC dimension of linear model = 3

Similarly according to VC dimension

VC dimension of RBF SVM = infinity

Theoretically, RBF SVM is more powerful but practically does not imply RBF SVM is the best in all cases.

NOTE: VC dimensions are a theoretical concept. They may not work well in real-world problems