Correlated predictors in a multiple regression model


#1

Hi Everyone,

I have learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant.

How does the variance of the regression coefficients get affected by including both predictors in the model or just having one?

How does including only one or including both predictors change the value/variance of my forecasted cost?

Thanks in advance for your help.


#2

Hi ,

Can anyone please help on my question ?


#3

If the the the model tries to fit the data with correlated variables then it will try to compensate each of the predictors for getting a valid outcome which will induce variance in the regression coefficients and hence a slight change in training data will produce amplified changes in the weighs. This might also make the model prone to overfitting as well.


#4

Hi @thulasi22ram

You mentioned linear model am I right ? So correlation is not good simple , I mean for the least square type of regression. What will happen? Your coefficient will have one large confidence interval and therefore your models are unstable… you can check this by using the VIF (Variance Inflation factor) it is the simplest way.
Hope this help.
Alain