Question on PCA and Factor Analysis/Regression?




Is PCA or Factor Analysis usage restricted to Survey data alone? Lot has been said about multicollinearlity effects on the regression. This is want I had inferred:

If the goal of the problem is to just predict or classify then multicollinearlity is not a problem as it does not affect significantly point or interval estimates.But if the goal is to profile the effect of predictor variables on the dependent variables then PCA is recommended.

Am I right making this statement. In real time projects, is PCA done each time before regression. I find that interpretation of the model can be tricky sometimes when PCA is done.

Balaji S.R



You are right about the first part - you don’t need to reduce the dimensions, if you are just predicting or classifying. However, if you are taking decision on the basis of relationships between variables, you should do a PCA.

The only scenario when you can leave PCA out is when you don’t see multi-collinearity / correlations in your variables. However, if you do see a strong or even moderate correlation between variables, PCA is highly suggested.