Main thing which comes while choosing algorithm for an ML problem is bias and variance. These are like proton and electron in an atom.

Bias : Difference between the expected and correct value which we are trying to predict using algorithm.

Variance : Degree to which model output varies between different test data iterations.(ie. for each test iteration with different data’s our model error rate increases)

Reason for high bias : Not properly choosing algorithm (leads to under-fitting).

How to reduce Bias.? Choose the correct algorithm such that it suits our use case and gives our expected output.

Reason for high variance : Overfitting (model is not generic, exactly fits training data)

How to reduce Variance.? Make sure train and test data’s come from the same distribution, eliminate outliers.

Only one thing we have to keep in mind, we cant achieve both low bias and low variance easily. Because when we try to reduce bias, variance will keep on increasing and vice versa.

Explanation about the above image :

Data will be divided into train and test sets and from their cross validation result we can get their error rates.

Examples of low-variance machine learning algorithms include: Linear Regression, Linear Discriminant Analysis and Logistic Regression.

Examples of high-variance machine learning algorithms include: Decision Trees, k-Nearest Neighbours and Support Vector Machines.

Examples of low-bias machine learning algorithms include: Decision Trees, k-Nearest Neighbours and Support Vector Machines.

Examples of high-bias machine learning algorithms include: Linear Regression, Linear Discriminant Analysis and Logistic Regression.