How does the variance-bias tradeoff in ensembles work

ensemble_methods

#1

Hello,

While learning about ensembles I understood that for effective ensembling,the base learner/s should have low bias and high variance.

1.Why is this the optimal situation?
2.What is meant by low bias here??
Can someone please help me with these??


#2

@pagal_guy,

Errors with any model can be broken down into three components Bias, Variance and Irreducible Error and this is important in an ensemble model. Bias error is used to measure how much on an average are the predicted values are different from the actual. A high bias error means we have a under-performing model which keeps on missing important trends (Underfitting). Variance on the other side measures how are the prediction made on same observation different from each other. A high variance model will over-fit on your training population and perform badly on any observation beyond training.

To improve the model performance, we increase the complexity of model and we come with reduction in error due to lower bias in the model but it works well till a point. As you continue to make your model more complex, you end up over-fitting your model and hence your model will start suffering from high variance. A better model should maintain a balance between these two types of errors. This is known as the trade-off management of bias-variance errors. Ensemble learning is one way to execute this trade off analysis.

Hope this helps!

Regards,
Imran