Feature Selection and Dimensionality Reduction

Whenever I search for “Feature Selection Techniques” I don’t see PCA in the replies. But I see that PCA is used in lot of research papers for feature selection. Is there any issue with PCA? Can someone make it clear for me?

@neeraj.bhopal83 , PCA is a dimensionality reduction technique.
Say if you have 1000s of features, you could perform PCA

  • To reduce it to 2 or 3 dimensions for visualizing
  • Or 20-30 features(which have the maximum variance) for training a model that wouldn’t be as computationally expensive.
    This article explains when PCA wouldn’t be suitable for feature selection -

The only way PCA is a valid method of feature selection is if the most important variables are the ones that happen to have the most variation in them. However this is usually not true.

PCA does not consider other factors like how the features might be correlated with the target variable. It just selects the features with maximum variance.
You can refer this for some good feature selection methods.

© Copyright 2013-2021 Analytics Vidhya