Is "bias" term in Machine Learning different from Deep Learning?




From what I have heard, bias in ML is related to underfitting and overfitting. Whereas bias in DL is a parameter. Are they different or one and the same?


Hi @albela_angur,

A good and truly an albela question. The intuition of bias in machine learning actually comes with bias vs. variance tradeoff which refers to choosing an optimal point without overfitting or underfitting a model.

The bias in the deep learning is a parameter in keras for a Dense layer. To be precise its use_bias and a boolean parameter. The parameter is boolean type which allows a to use a bias vector. Its mostly used to shift the activation function and the decision split. A simple example - If activation function splits a class into 1 or 0 based on if probability > 0.5 = 1 else 0.

Now, with use_bias = True the model will use another weight wb that shifts the split to if probability > 0.7 = 1 else 0 after its trained.

Hope this answered your question. Have a great day.