Predictive power of regression models?

machine_learning

#1

Hi Experts,

How do you measure predictive power of regression models?

Regards,
Tony


#2

@tillutony- It is very important what do you mean by the predictive power of the model usually we always focused on reducing training error but this, not a good practice we always try to reduce the test error .But how we can about the test error while building a model on training data set.

There are some parameters which can help are as follows
1-most important one is cross-validation
2-Adjusted R^2
3-cp
4-AIC

Hope this helps!

Regards,
Hinduja


#3

Any pointers on

Root Mean Squared Error (RMSE)


#4

Hi @tillutony

For linear regression, this is a good measure to get a good understanding how the model behaves, for non linear give some information but does not relate to normal distribution therefor the next line should be ignored.
The RMSE uses a formula similar to OLS, and then knowing that for linear regression the response (Y) should follow a normal distribution (assumption, the residual must be normally distributed ) , RMSE will tell you that you have probability of .68 of you predictions to be in the RMSE (+_) and then .98 to be in two RMSE. As the RMSE is in the same unit as the response it gives you one intuitive understanding of the “quality” of you model. Keep in mind this comment is valid for linear regression form. .
For the model side R2 (explanation of variance) is better and AIC is bounded with complexity and there more difficult to grasp for me anyway :slight_smile: but for non linear is good specially when you start to have lot of variables.
Hope this help.
Alain